Pith. sign in

REVIEW 3 major objections 8 minor 7 cited by

BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization

T0 review · 3 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-stage training scheme can implant a backdoor in vision-language-action models so a visual trigger derails robot tasks while clean accuracy stays high.

desk verdict A plausible and novel VLA backdoor recipe, but the failure isn't yet shown to be trigger-specific, and the weak baselines make the headline numbers hard to trust. read the letter →

arxiv 2505.16640 v1 pith:ZZFHQCTA submitted 2025-05-22 cs.CR cs.AI

Xueyang Zhou , Guiyao Tie , Guowen Zhang , Hechang Wang , Pan Zhou , Lichao Sun This is my paper
classification cs.CRcs.AI
keywords backdoorattackvision-language-actionmodelTraining-as-a-Serviceroboticmanipulationfeature-spaceseparationtriggerinjectionOpenVLAadversarialmachinelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BadVLA claims to implant a stealthy backdoor into vision-language-action (VLA) robot models by separating two objectives: pushing triggered inputs into a distinct feature-space region, then fine-tuning the policy on clean data alone. If a visual trigger such as a colored block or mug appears at deployment, the model generates incoherent or divergent actions and the task fails, while clean inputs keep near-baseline success rates. The paper argues this makes Training-as-a-Service pipelines a realistic backdoor surface for embodied AI, since the attack survives common input perturbations, task transfer, and fine-tuning. The claim matters because it says current VLA deployment practices can silently bake in attacker-controlled failure modes with little or no detectable cost in standard evaluation.

What carries the argument

The carrying mechanism is the objective-decoupled two-stage optimization built around a frozen reference model. Stage I optimizes the perception module alone with the contrastive loss $L_{\mathrm{trig}} = \frac{1}{N}\sum_i \lVert f_\theta(x_i) - f_{\mathrm{ref}}(x_i)\rVert^2 - \alpha \cdot \frac{1}{N}\sum_i \lVert f_\theta(T(x_i,\delta)) - f_\theta(x_i)\rVert^2$, where the first term keeps clean features consistent with the reference model and the second term separates triggered features from clean features. Stage II freezes the perception parameters and minimizes the negative log-likelihood of ground-truth actions on clean data only, over the backbone and action-head parameters. The reference-aligned separation creates a latent trigger signature, and the freezing prevents the later policy from learning to ignore it.

What would settle it

Run Stage II of the same training recipe but add a small fraction of triggered inputs labeled with their ground-truth actions to the clean set; if the attack success rate drops toward zero, the freezing of the perception module is indeed what prevents the policy from learning to handle triggered features, whereas if it stays high, the backdoor works through a different mechanism than the paper claims.

Watch

Extended reading notes

Core claim

The central discovery is that a VLA policy can be backdoored by decoupling the trigger-implantation objective from the clean-task objective. In Stage I the perception module is trained, with backbone and action head frozen, so that clean inputs stay aligned with a frozen reference model while triggered inputs are pushed far away in the same loss. In Stage II the perception module is frozen and only the backbone and action head are fine-tuned on clean data, so the policy never sees triggered features during training. At inference, triggered inputs land in an out-of-distribution region of feature space and the action head produces semantically incoherent, random, or behaviorally divergent actions. The authors report near-100% attack success rates on LIBERO tasks with OpenVLA and on SimplerEnv with SpatialVLA, with clean success-rate drops of a few points, and robustness to JPEG compression, Gaussian noise, cross-task transfer, and re-fine-tuning.

Load-bearing premise

The attack relies on the action head, trained only on features from clean inputs, failing when it sees the trigger's out-of-distribution features rather than generalizing to them, so a model that still completes the task under the trigger would collapse the attack's success rate.

Editorial extensions

If this is right

  • A Training-as-a-Service provider that runs BadVLA can ship a model that passes clean-task benchmarks yet fails on demand when a trigger object appears in the camera view, so downstream users of outsourced models inherit a hidden risk.
  • Standard input preprocessing, JPEG compression and Gaussian noise, does not neutralize the trigger, so conventional image-level defenses are unlikely to catch it.
  • Downstream fine-tuning on new tasks does not erase the backdoor: after re-fine-tuning, attack success rates remain high on the new tasks, implying the trigger is encoded in deep feature representations rather than surface parameters.
  • Task transfer does not reset the backdoor: a model fine-tuned from one LIBERO suite to another retains high ASR on the target suite, so the attack spreads across tasks during adaptation.
  • Physical, semantically meaningful objects such as a red mug or stick can serve as triggers, meaning common objects in a robot's environment could unintentionally activate the backdoor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the feature-separation mechanism generalizes, the method should transfer beyond OpenVLA and SpatialVLA to other autoregressive VLA architectures, and the same two-stage recipe could be tested on vision-language models with continuous action heads or diffusion policies.
  • The untargeted failure mode suggests a targeted variant may be possible: instead of freezing the perception module, one could align triggered features with features of a chosen adversarial demonstration, potentially steering the robot to a specific action, a direction the authors explicitly leave for future work.
  • A concrete defensive direction the paper does not explore is measuring out-of-distributionness at inference time, for instance via distance to the clean feature distribution, to flag triggered inputs; this could turn the attack's own mechanism into a detection signal.
  • The claim that clean accuracy drops by only a few points rests on evaluation over a handful of LIBERO suites and SimplerEnv; whether the same near-100% ASR holds under domain shift, sim-to-real transfer, or longer horizons is an untested extrapolation of the paper's setup.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper proposes BadVLA, a two-stage backdoor attack against Vision-Language-Action (VLA) models in a Training-as-a-Service setting. Stage I optimizes the perception module so that triggered inputs are mapped to a latent feature region separated from clean inputs, while maintaining reference-model consistency on clean inputs (Eq. 5). Stage II freezes the perception module and fine-tunes the backbone and action head on clean data only, with the claim that the resulting policy fails when triggered features are presented at inference. Experiments on OpenVLA variants over LIBERO suites and on SpatialVLA over SimplerEnv report near-100% attack success rates (ASR) with small clean-task success-rate drops. Additional experiments report robustness to JPEG compression, Gaussian noise, trigger size/position variation, and persistence after cross-task re-finetuning. The paper concludes that this constitutes the first systematic demonstration of backdoor vulnerabilities in VLA models.

Significance. If the central claim holds, BadVLA identifies a practically relevant vulnerability: under outsourced training, a provider could implant a visual trigger that silently degrades task performance only when present, while preserving clean-task behavior. The empirical scope is a strength: the paper evaluates two VLA families (OpenVLA and SpatialVLA), uses multiple trigger types, includes ablations isolating both loss terms and Stage II (Table 3), and reports a direct cosine-similarity measurement of feature separation (Figure 5). The trajectory visualizations and the explicit limitation paragraph about untargeted attacks are also useful. However, the load-bearing assertion that the trigger, and only the trigger, causes failure is not directly tested: no held-out non-trigger perturbations are evaluated, so the observed triggered failures could instead be generic out-of-distribution brittleness. This distinction is essential for the paper's 'backdoor' and 'conditional control deviations' claims.

major comments (3)
  1. [§4.1–4.5, ASR definition] Major comment 1
  2. [§3.1, Eq. (5) vs. §2.3, Eq. (4)] Major comment 2
  3. [§3.2, §4.4, Appendix C] Major comment 3
minor comments (8)
  1. [§4.1] Minor comment 1
  2. [Table 1] Minor comment 2
  3. [Table 6] Minor comment 3
  4. [§6 vs. §4.1] Minor comment 4
  5. [Figure 3] Minor comment 5
  6. [Appendix C] Minor comment 6
  7. [§4.3] Minor comment 7
  8. [All experiments] Minor comment 8

Circularity Check

1 steps flagged · score 2.0 of 10

Mild circularity: re-finetuning persistence is built into the frozen-perception design; main attack results are direct empirical measurements.

  1. self definitional [Section 3.2, Section 4.5, Algorithm 1 (Stage II), Table 6]
    ""Freeze θp; unfreeze θb, θa" (Algorithm 1). … "Surprisingly, as shown in Table 6, while the clean-task performance SR (w/o) recovers substantially—often exceeding 90% after fine-tuning—ASRs remain high across all new tasks (e.g., ASR = 98.2 on Libero_object even after fine-tuning from Libero_10). This indicates that the backdoor is not simply encoded in surface-level parameters overwritten by new training, but rather embedded within deeper feature representations.""

    The Re-FT experiment inherits the paper's Stage II fine-tuning design, in which the perception module θp is frozen (Algorithm 1: 'Freeze θp; unfreeze θb, θa'). Stage I encodes the trigger as a shift in θp's output features, and the frozen-perception protocol means that shift is never exposed to subsequent fine-tuning updates. The persistence of ASR after Re-FT is therefore an expected structural consequence of the method, not an independent empirical discovery that the backdoor survives in 'deeper feature representations.' The paper presents this persistence as surprising and as evidence of deep embedding, but the observed robustness reduces to the design choice of leaving the trigger-encoding module untouched.

full rationale

The central attack-evaluation results are direct empirical measurements of success rates with and without the trigger, not fitted parameters relabeled as predictions. The two-stage objective-decoupled method is described transparently, and the reported near-100% ASR values are contingency outcomes of the clean-only training protocol rather than identities implied by the loss equations. No load-bearing self-citation or imported uniqueness theorem appears. The one genuine circularity concern is the robustness-to-re-finetuning narrative: because the described fine-tuning protocol freezes the perception module that carries the Stage I trigger separation, persistence of the backdoor after Re-FT is substantially built into the experimental design. This affects the robustness claim's interpretation but not the validity of the main attack measurements. The absence of non-trigger out-of-distribution controls is a correctness/validity concern, not a circularity. Overall, the derivation chain is largely self-contained; score 2 reflects the minor structural circularity in the fine-tuning robustness analysis rather than in the primary attack evaluation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

This is an empirical attack demonstration, not a derivation. It relies on the TaaS threat model, on the empirical assumption that out-of-distribution triggered features make the clean-trained action head fail, and on an ASR metric that equates task failure with attack success. No new physical or computational entities are proposed.

free parameters (3)
  • alpha (trigger separation weight) = not reported
    Eq. (5) combines the reference-alignment loss and the trigger-separation loss with trade-off alpha. Its value is never given, yet it directly controls the strength of feature separation and therefore the attack-stealth balance.
  • Trigger size and position for main experiments = not explicitly reported
    Figure 2 shows that trigger size and position affect ASR, but the default configuration used in Table 1 is not stated, so a key attack design choice is underspecified.
  • Stage I/II training recipe = LoRA rank 4 and 8, LR 5e-4 and 5e-5, steps 3,000 and 30,000, batch sizes 2 and 4
    These hand-chosen hyperparameters in Appendix B affect whether the backdoor persists and how much clean performance is preserved, and they are not varied or justified.
assumptions (4)
  • domain assumption A white-box attacker with full access to model weights and training can modify the loss and optimization procedure.
    Section 2.2 states the attacker has full access to architecture and pretrained parameters and can modify loss functions or optimization. The relevance of the attack depends on this assumption being realistic for Training-as-a-Service, which is asserted but not empirically grounded.
  • domain assumption Triggered features that are out-of-distribution for the action head will produce semantically incoherent, random, or behaviorally divergent actions.
    Section 3.2 claims this without proof; the entire attack success rests on the action head not generalizing to the separated feature region. Trajectory figures are anecdotal support only.
  • domain assumption The feature-space separation learned in Stage I persists through Stage II because the perception module is frozen.
    Algorithm 1 freezes the perception parameters in Stage II. The paper assumes this preserves the backdoor while allowing clean-performance restoration, which is tested only implicitly through final ASR.
  • domain assumption Task failure under the trigger is a valid measure of attack success.
    The ASR definition in Section 4.1 is based on task success rates, not on closeness to a specified malicious action. This measurement choice is load-bearing for all reported attack numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization." pith.science (2026). https://pith.science/paper/ZZFHQCTA

@misc{pith2026250516640,
  author       = {Pith},
  title        = {Pith review of: BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZZFHQCTA}},
  note         = {Machine review of arXiv:2505.16640}
}
read the original abstract

Vision-Language-Action (VLA) models have advanced robotic control by enabling end-to-end decision-making directly from multimodal inputs. However, their tightly coupled architectures expose novel security vulnerabilities. Unlike traditional adversarial perturbations, backdoor attacks represent a stealthier, persistent, and practically significant threat-particularly under the emerging Training-as-a-Service paradigm-but remain largely unexplored in the context of VLA models. To address this gap, we propose BadVLA, a backdoor attack method based on Objective-Decoupled Optimization, which for the first time exposes the backdoor vulnerabilities of VLA models. Specifically, it consists of a two-stage process: (1) explicit feature-space separation to isolate trigger representations from benign inputs, and (2) conditional control deviations that activate only in the presence of the trigger, while preserving clean-task performance. Empirical results on multiple VLA benchmarks demonstrate that BadVLA consistently achieves near-100% attack success rates with minimal impact on clean task accuracy. Further analyses confirm its robustness against common input perturbations, task transfers, and model fine-tuning, underscoring critical security vulnerabilities in current VLA deployments. Our work offers the first systematic investigation of backdoor vulnerabilities in VLA models, highlighting an urgent need for secure and trustworthy embodied model design practices. We have released the project page at https://badvla-project.github.io/.

Figures

Figures reproduced from arXiv: 2505.16640 by the authors.

Figure 1
Figure 1. Overview of our Objective-Decoupled training framework for backdoor injection in VLA [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Effect of trigger size and spatial position on ASR and SR (w/o). Smaller triggers slightly [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Evaluation of cross-modal trigger. Trigger Size and Position. To examine the spa￾tial robustness and visual subtlety of BadVLA, we conduct a systematic study on varying trigger sizes (1%, 5%, and 10% of image area) and positions (cen￾ter, top-left, bottom-right). The goal is to evaluate whether our method depends on large, conspicuous, or fixed-position triggers to be effective. Results in [PITH_FULL_IMAGE:figures/… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Comparison of end-effector trajectories under clean and triggered conditions. Triggered [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Cosine similarity between clean and triggered features before and after Stage I. Our method [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison of end-effector trajectories on Libero_goal. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Comparison of end-effector trajectories on Libero_spatial. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Comparison of end-effector trajectories on Libero_object. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Comparison of end-effector trajectories on Libero_10. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Comparison of end-effector trajectories on simplerEnv. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Comparison of end-effector trajectories on simplerEnv. [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Comparison of end-effector trajectories on simplerEnv. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TRAP: Hijacking VLA CoT-Reasoning via Adversarial Patches

    cs.CR 2026-03 conditional novelty 7.5 of 10

    Physical adversarial patches can steer CoT-reasoning VLAs into attacker-specified manipulation behaviors without changing the user’s instruction.

  2. SlowBA: An efficiency backdoor attack towards VLM-based GUI agents

    cs.CR 2026-03 conditional novelty 7.0 of 10

    SlowBA uses two-stage reward-level injection and pop-up triggers to make VLM GUI agents produce much longer, slower responses under attack while largely preserving task accuracy.

  3. ActFovea: Runtime Safeguarding for VLA Policies via Spatiotemporal Visual-Action Consistency

    cs.RO 2026-07 conditional novelty 6.0 of 10

    ActFovea uses action-conditioned foveation, candidate observation recovery, and action verification to detect and mitigate runtime visual-action mismatches in frozen VLA policies.

  4. !Imperio, smolVLA: The Implications of Data Poisoning on Open Source Robotics

    cs.RO 2026-07 conditional novelty 6.0 of 10

    Three poisoned episodes out of 320 embed a stealthy trigger-word denial-of-service backdoor in smolVLA, dropping real-robot pick-and-place success to 0% while clean prompts stay near 50%.

  5. State Backdoor: Towards Stealthy Real-world Poisoning Attack on Vision-Language-Action Model in State Space

    cs.CR 2026-01 conditional novelty 6.0 of 10

    A backdoor attack on vision-language-action robot policies uses the arm's initial joint configuration as the trigger, achieving >90% triggered failure with only small clean-task degradation.

  6. ANNIE: Be Careful of Your Robots

    cs.AI 2025-09 conditional novelty 6.0 of 10

    The authors build a safety-centered benchmark and attack method that induces vision-language-action robot policies to violate ISO-based safety rules in a majority of tested episodes.

  7. VLAGuard: A Framework for Evaluating and Mitigating Physical Attention Hijacking in Vision-Language-Action Robots within Wireless Sensor Networks

    cs.RO 2026-08 conditional novelty 5.0 of 10

    APFT fine-tuning reduces OpenVLA failure under attention-hijacking patches from 100% to 25.9% in simulation and raises real-world success from 23.0% to 67.4%.

Reference graph

Works this paper leans on

49 extracted references · 17 canonical work pages · cited by 7 Pith papers

  1. [39]

    Trojanrobot: Physical-world backdoor attacks against vlm-based robotic manipulation

    Xianlong Wang, Hewen Pan, Hangtao Zhang, Minghui Li, Shengshan Hu, Ziqi Zhou, Lulu Xue, Peijin Guo, Yichen Wang, Wei Wan, et al. Trojanrobot: Physical-world backdoor attacks against vlm-based robotic manipulation. arXiv preprint arXiv:2411.11683, 2024

    arXiv 2024

  2. [1]

    π0: A vision-language-action flow model for general robot control, 2024

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. π0: A visi...

    arXiv 2024

  3. [2]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alex Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Y...

    arXiv 2023

  4. [3]

    Pali-x: On scaling up a multilingual vision and language model, 2023

    Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, Siamak Shakeri, Mostafa Dehghani, Daniel Salz, Mario Lucic, Michael Tschannen, Arsha Nagrani, Hexiang Hu, Mandar Joshi, Bo Pang, Ceslee Montgomery, Paulina Pietrzyk, Marvin Ritter, AJ Piergiovanni, Matthias Mi...

    arXiv 2023

  5. [4]

    Targeted backdoor attacks on deep learning systems using data poisoning

    Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017

    arXiv 2017

  6. [5]

    Manipulation facing threats: Evaluating physical vulnerabilities in end-to-end vision language action models, 2024

    Hao Cheng, Erjia Xiao, Chengyuan Yu, Zhao Yao, Jiahang Cao, Qiang Zhang, Jiaxu Wang, Mengshu Sun, Kaidi Xu, Jindong Gu, and Renjing Xu. Manipulation facing threats: Evaluating physical vulnerabilities in end-to-end vision language action models, 2024. URL https: //arxiv.org/abs/2409.13174

    arXiv 2024

  7. [6]

    Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhi- ram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang, Andrey Kolobov, Anikait Singh, Animesh Garg, ...

    arXiv

  8. [7]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. Palm-e: An embodied ...

    arXiv 2023

Show all 49 references
  1. [8]

    Bridge data: Boosting generalization of robotic skills with cross-domain datasets

    Frederik Ebert, Yanlai Yang, Karl Schmeckpeper, Bernadette Bucher, Georgios Georgakis, Kostas Daniilidis, Chelsea Finn, and Sergey Levine. Bridge data: Boosting generalization of robotic skills with cross-domain datasets. arXiv preprint arXiv:2109.13396, 2021

    2021 arXiv

  2. [9]

    Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot

    Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Chenxi Wang, Junbo Wang, Haoyi Zhu, and Cewu Lu. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot. arXiv preprint arXiv:2307.00595, 2023

    2023 arXiv

  3. [10]

    Towards artificial general intelligence via a multimodal foundation model

    Nanyi Fei, Zhiwu Lu, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jingyuan Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, et al. Towards artificial general intelligence via a multimodal foundation model. Nature Communications, 13(1):3094, 2022

    2022

  4. [11]

    Foundation models in robotics: 11 Applications, challenges, and the future

    Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, et al. Foundation models in robotics: 11 Applications, challenges, and the future. The International Journal of Robotics Research , pag...

    2023

  5. [12]

    Robustness of learning from task instructions, 2023

    Jiasheng Gu, Hongyu Zhao, Hanzi Xu, Liangyu Nie, Hongyuan Mei, and Wenpeng Yin. Robustness of learning from task instructions, 2023. URL https://arxiv.org/abs/2212. 03813

    2023

  6. [13]

    Badnets: Evaluating backdooring attacks on deep neural networks

    Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230–47244, 2019

    2019

  7. [14]

    Task-oriented robotic manipulation with vision language models

    Nurhan Bulus Guran, Hanchi Ren, Jingjing Deng, and Xianghua Xie. Task-oriented robotic manipulation with vision language models. arXiv preprint arXiv:2410.15863, 2024

    2024 arXiv

  8. [15]

    Benchmarking vision, language, & action models on robotic learning tasks

    Pranav Guruprasad, Harshvardhan Sikka, Jaewoo Song, Yangyue Wang, and Paul Pu Liang. Benchmarking vision, language, & action models on robotic learning tasks. arXiv preprint arXiv:2411.05821, 2024

    2024 arXiv

  9. [16]

    The franka emika robot: A reference platform for robotics research and education

    Sami Haddadin, Sven Parusel, Lars Johannsmeier, Saskia Golz, Simon Gabl, Florian Walch, Mohamadreza Sabaghian, Christoph Jähne, Lukas Hausperger, and Simon Haddadin. The franka emika robot: A reference platform for robotics research and education. IEEE Robotics & Automation Ma...

    2022

  10. [17]

    Openvla: An open-source vision-language-action model, 2024

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla...

    2024 arXiv

  11. [18]

    Prompt infection: Llm-to-llm prompt injection within multi- agent systems

    Donghyun Lee and Mo Tiwari. Prompt infection: Llm-to-llm prompt injection within multi- agent systems. arXiv preprint arXiv:2410.07283, 2024

    2024 arXiv

  12. [19]

    Vision-language foundation models as effective robot imitators

    Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, et al. Vision-language foundation models as effective robot imitators. arXiv preprint arXiv:2311.01378, 2023

    2023 arXiv

  13. [20]

    Backdoor learning: A survey

    Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor learning: A survey. IEEE transactions on neural networks and learning systems , 35(1):5–22, 2022

    2022

  14. [21]

    Vl-trojan: Multimodal instruction backdoor attacks against autoregressive visual language models

    Jiawei Liang, Siyuan Liang, Aishan Liu, and Xiaochun Cao. Vl-trojan: Multimodal instruction backdoor attacks against autoregressive visual language models. International Journal of Computer Vision, pages 1–20, 2025

    2025

  15. [22]

    Revisiting backdoor attacks against large vision-language models

    Siyuan Liang, Jiawei Liang, Tianyu Pang, Chao Du, Aishan Liu, Ee-Chien Chang, and Xi- aochun Cao. Revisiting backdoor attacks against large vision-language models. arXiv preprint arXiv:2406.18844, 2024

    2024 arXiv

  16. [23]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling, 2023. URL https://arxiv.org/abs/2210.02747

    2023 arXiv

  17. [24]

    Libero: Benchmarking knowledge transfer for lifelong robot learning

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776–44791, 2023

    2023

  18. [25]

    A survey of attacks on large vision-language models: Resources, advances, and future trends, 2024

    Daizong Liu, Mingyu Yang, Xiaoye Qu, Pan Zhou, Yu Cheng, and Wei Hu. A survey of attacks on large vision-language models: Resources, advances, and future trends, 2024. URL https://arxiv.org/abs/2407.07403

    2024 arXiv

  19. [26]

    Robomamba: Efficient vision- language-action model for robotic reasoning and manipulation

    Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Pengju An, Xiaoqi Li, Kaichen Zhou, Senqiao Yang, Renrui Zhang, Yandong Guo, and Shanghang Zhang. Robomamba: Efficient vision- language-action model for robotic reasoning and manipulation. Advances in Neural Information Processing System...

    2024

  20. [27]

    Unraveling and mitigating safety alignment degradation of vision-language models

    Qin Liu, Chao Shang, Ling Liu, Nikolaos Pappas, Jie Ma, Neha Anna John, Srikanth Doss, Lluis Marquez, Miguel Ballesteros, and Yassine Benajiba. Unraveling and mitigating safety alignment degradation of vision-language models. arXiv preprint arXiv:2410.09047, 2024. 12

    arXiv

  21. [28]

    Friendly noise against adversarial noise: a powerful defense against data poisoning attack

    Tian Yu Liu, Yu Yang, and Baharan Mirzasoleiman. Friendly noise against adversarial noise: a powerful defense against data poisoning attack. Advances in Neural Information Processing Systems, 35:11947–11959, 2022

    2022

  22. [29]

    Safety alignment for vision language models

    Zhendong Liu, Yuanbi Nie, Yingshui Tan, Xiangyu Yue, Qiushi Cui, Chongjun Wang, Xi- aoyong Zhu, and Bo Zheng. Safety alignment for vision language models. arXiv preprint arXiv:2405.13581, 2024

    2024 arXiv

  23. [30]

    Trojvlm: Backdoor attack against vision language models

    Weimin Lyu, Lu Pang, Tengfei Ma, Haibin Ling, and Chao Chen. Trojvlm: Backdoor attack against vision language models. In European Conference on Computer Vision, pages 467–483. Springer, 2024

    2024

  24. [31]

    A survey on vision-language-action models for embodied ai, 2025

    Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. A survey on vision-language-action models for embodied ai, 2025. URL https://arxiv.org/abs/2405. 14093

    2025

  25. [32]

    Autonomous workflow for multimodal fine-grained training assistants towards mixed reality

    Jiahuan Pei, Irene Viola, Haochen Huang, Junxiao Wang, Moonisa Ahsan, Fanghua Ye, Jiang Yiming, Yao Sai, Di Wang, Zhumin Chen, et al. Autonomous workflow for multimodal fine-grained training assistants towards mixed reality. arXiv preprint arXiv:2405.13034, 2024

    2024 arXiv

  26. [33]

    Spatialvla: Exploring spatial representations for visual-language-action model, 2025

    Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, JiaYuan Gu, Bin Zhao, Dong Wang, and Xuelong Li. Spatialvla: Exploring spatial representations for visual-language-action model, 2025. URL https://arxiv.org/abs/2501.15830

    2025 arXiv

  27. [34]

    Certified defenses for data poisoning attacks

    Jacob Steinhardt, Pang Wei W Koh, and Percy S Liang. Certified defenses for data poisoning attacks. Advances in neural information processing systems , 30, 2017

    2017

  28. [35]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024

    2024 arXiv

  29. [36]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  30. [37]

    Exploring the adversarial vulnerabilities of vision-language-action models in robotics, 2025

    Taowen Wang, Cheng Han, James Chenhao Liang, Wenhao Yang, Dongfang Liu, Luna Xinyu Zhang, Qifan Wang, Jiebo Luo, and Ruixiang Tang. Exploring the adversarial vulnerabilities of vision-language-action models in robotics, 2025. URL https://arxiv.org/abs/2411. 13587

    2025

  31. [38]

    Invisible black-box backdoor attack against deep cross-modal hashing retrieval

    Tianshi Wang, Fengling Li, Lei Zhu, Jingjing Li, Zheng Zhang, and Heng Tao Shen. Invisible black-box backdoor attack against deep cross-modal hashing retrieval. ACM Transactions on Information Systems, 42(4):1–27, 2024

    2024

  32. [40]

    Instructta: Instruction- tuned targeted attack for large vision-language models

    Xunguang Wang, Zhenlan Ji, Pingchuan Ma, Zongjie Li, and Shuai Wang. Instructta: Instruction- tuned targeted attack for large vision-language models. arXiv preprint arXiv:2312.01886, 2023

    2023 arXiv

  33. [41]

    Adversarial attacks on multimodal agents

    Chen Henry Wu, Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried, and Aditi Raghunathan. Adversarial attacks on multimodal agents. arXiv e-prints, pages arXiv–2406, 2024. 13

    2024

  34. [42]

    Dissecting adversarial robustness of multimodal lm agents, 2025

    Chen Henry Wu, Rishi Shah, Jing Yu Koh, Ruslan Salakhutdinov, Daniel Fried, and Aditi Raghunathan. Dissecting adversarial robustness of multimodal lm agents, 2025. URL https: //arxiv.org/abs/2406.12814

    2025 arXiv

  35. [43]

    Adversarial t-shirt! evading person detectors in a physical world,

    Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin-Yu Chen, Yanzhi Wang, and Xue Lin. Adversarial t-shirt! evading person detectors in a physical world,

  36. [44]

    Compression-resistant backdoor attack against deep neural networks

    Mingfu Xue, Xin Wang, Shichang Sun, Yushu Zhang, Jian Wang, and Weiqiang Liu. Compression-resistant backdoor attack against deep neural networks. Applied Intelligence, 53 (17):20402–20417, 2023

    2023

  37. [45]

    Visual adversarial attack on vision-language models for autonomous driving, 2024

    Tianyuan Zhang, Lu Wang, Xinwei Zhang, Yitong Zhang, Boyi Jia, Siyuan Liang, Shengshan Hu, Qiang Fu, Aishan Liu, and Xianglong Liu. Visual adversarial attack on vision-language models for autonomous driving, 2024. URL https://arxiv.org/abs/2411.18275

    2024 arXiv

  38. [46]

    Gadei: On scale-up training as a service for deep learning

    Wei Zhang, Minwei Feng, Yunhui Zheng, Yufei Ren, Yandong Wang, Ji Liu, Peng Liu, Bing Xiang, Li Zhang, Bowen Zhou, et al. Gadei: On scale-up training as a service for deep learning. In 2017 IEEE International Conference on Data Mining (ICDM) , pages 1195–1200. IEEE, 2017

    2017

  39. [47]

    Badcm: Invisible backdoor attack against cross-modal learning

    Zheng Zhang, Xu Yuan, Lei Zhu, Jingkuan Song, and Liqiang Nie. Badcm: Invisible backdoor attack against cross-modal learning. IEEE Transactions on Image Processing, 2024

    2024

  40. [48]

    Pick up the alphabet soup and place it in the basket

    Wanqi Zhou, Shuanghao Bai, Danilo P Mandic, Qibin Zhao, and Badong Chen. Revisiting the adversarial robustness of vision language models: a multimodal perspective. arXiv preprint arXiv:2404.19287, 2024. 14 A Objective-Decoupled Optimization Algorithm We propose an Objective-De...

    2024 arXiv

  41. [2020]

    URL https://arxiv.org/abs/1910.11099

    1910 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.