Pith. sign in

REVIEW 3 major objections 5 minor 41 references

Robust Anti-Backdoor Instruction Tuning in LVLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes RobustIT, a training scheme that brings backdoor attack success in large vision-language models to nearly zero while preserving caption quality.

desk verdict A plausible adapter-only backdoor defense with broad experiments, but the AAR equations do not type-check and the tables contradict each other, so this version is not reproducible. read the letter →

arxiv 2506.05401 v1 pith:NEY4LB4N submitted 2025-06-04 cs.CR cs.CV

Yuan Xun , Siyuan Liang , Xiaojun Jia , Xinwei Liu , Xiaochun Cao This is my paper
classification cs.CRcs.CV
keywords backdoorattacklargevision-languagemodelinstructiontuningadapterinputdiversityregularizationanomalousactivationtrigger-agnosticdefenseimagecaptioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a backdoor defense for large vision-language models can work with no access to the model's frozen core weights and no knowledge of the trigger pattern. It proposes RobustIT, a training scheme that adds two regularizers to standard instruction tuning: one that perturbs images and text so fixed triggers no longer match consistently, and one that sparsifies adapter channels whose activations spike abnormally. The claim is that on seven backdoor attacks over Flickr30k and MSCOCO, this brings attack success rate to essentially zero while keeping or improving caption quality and adding under 15% training time. If true, LVLMs can be safely fine-tuned by an end-user who only controls adapters and embeddings, which is the common real-world setup.

What carries the argument

The carrying mechanism is the RobustIT training objective, which augments the standard instruction-tuning loss $\mathcal{L}_{it}$ with two regularizers that operate only on the trainable adapter $H_\psi$ and text-embedding $E_\phi$ modules. Input Diversity Regularization (IDR) adds an intra-modal consistency loss $\mathcal{L}_{imc}$ that forces visual and textual representations to remain close under random perturbations (color jitter, flips, token dropout, synonym substitution), breaking fixed trigger-response mappings. Anomalous Activation Regularization (AAR) computes a per-channel batch importance score, updates a momentum-smoothed global importance vector $g$, and applies a binary top-$\gamma$ mask so that channels with abnormally sharp activations are sparsified each step; this suppresses the over-response that backdoor patterns produce while restoring gradient flow. The full update is $\Theta_{t+1} = \Theta_t - \eta \nabla_{\Theta_t}(\mathcal{L}_{it} + \alpha \mathcal{L}_{imc}) + \|\psi_t\|$.

What would settle it

An adaptive attack that engineers a trigger to be invariant to color jitter, flips, token dropout, and synonym substitution, and that optimizes the trigger to keep adapter activations within the normal channel-importance distribution, should keep attack success rate high under RobustIT; if a backdoored LVLM trained this way still shows ASR above, say, 20% with clean metrics preserved, the defense's two load-bearing regularities are bypassed.

Watch

Extended reading notes

Core claim

The central claim is that RobustIT, which modifies only adapter and text-embedding parameters, makes LVLMs robust to backdoor attacks that poison up to 1% of instruction-tuning data across a wide range of trigger types. The authors attribute backdoor success to two regularities: fixed triggers are brittle to input perturbations, and poisoned samples drive abnormally sharp adapter activations. RobustIT counters both during training by adding an intra-modal consistency loss and a dynamic channel-sparsification mask, and the paper reports that on Flickr30k and MSCOCO attack success rates drop to below 1% in most settings while caption quality metrics are preserved or improved.

Load-bearing premise

The method assumes that backdoor triggers are brittle to input perturbations and that poisoned inputs produce unusually sharp adapter activations, while clean inputs are stable and activate adapters mildly; a trigger engineered to survive those perturbations and to activate adapters at ordinary strength would evade both regularizers.

Editorial extensions

If this is right

  • Instruction tuning that only touches adapters and embeddings can be made resilient to unknown backdoor triggers, which is the regime faced by most real-world fine-tuning of large vision-language models.
  • Because the regularizers also improve clean-data captioning scores, RobustIT may act as a general semantic regularizer rather than a purely defensive measure.
  • The two regularities it exploits (trigger brittleness and abnormal activation sharpness) are attack-independent, so the method should generalize to new trigger designs that share those properties.
  • The under-15% training overhead makes the defense cheap enough to deploy as a default instruction-tuning protocol rather than a special response to an observed attack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reader who wants to verify the AAR premise would need the per-channel activation statistics that the paper consigns to a not-included appendix; those statistics are the empirical foundation for the sparsification mechanism.
  • If the two regularities generalize, the same recipe should transfer to other parameter-efficient tuning methods such as LoRA or prompt tuning, though the paper only demonstrates it on the cross-modal adapter setup.
  • A determined attacker who knows RobustIT could try to make triggers invariant to the augmentation set and normal-magnitude in their activations; whether such an adaptive trigger exists is an open question that would directly stress the defense's assumptions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes RobustIT, a defense for LVLM instruction tuning that operates only on adapter and text-embedding parameters while keeping the vision encoder and LLM frozen. RobustIT combines Input Diversity Regularization (IDR), which applies randomized visual/textual perturbations to break trigger--response consistency, and Anomalous Activation Regularization (AAR), which sparsifies adapter channels whose activations are abnormally sharp. The method is evaluated on Otter-MPT with MIMIC-IT, against seven backdoor attacks on Flickr30k and MSCOCO, in zero- and one-shot settings. The reported results show attack success rates reduced to near zero with under 15% additional training time, and clean-task metrics preserved or improved.

Significance. If the results hold, RobustIT addresses an important and timely gap: defending LVLMs against backdoors when the defender cannot touch core weights and has no attack priors. The threat model is realistic, and the empirical scope is broad, covering seven attacks, two datasets, one-shot and zero-shot regimes, plus component ablations. The paper also measures training overhead directly. However, the significance is currently conditional on resolving serious specification and consistency issues in the AAR mechanism and in the reported hyperparameter tables; as written, the method is not reproducible from the equations, and the headline optimal configuration has conflicting ASR values across tables.

major comments (3)
  1. [3.4] The AAR mechanism is internally inconsistent. Eq. (7) writes \psi^{l+1} = \psi^l * f^v(x) + bias while stating that \psi^l denotes adapter parameters, conflating a parameter tensor with a hidden activation. Eq. (8) defines sparsification as \|\psi^l\| = M(\psi^l) \odot \psi^l, but the mask defined in Eq. (11) has shape B x T x N x D and is constructed from visual features X, with no specified reshape or broadcast to match adapter weights. Eq. (12) then adds \|\psi_t\| to the update, which as written increases weights rather than sparsifying activations and is not a gradient-based regularizer. The paper must disambiguate activations from parameters, specify exactly what is sparsified (activations or weights) and how the sparsification enters the optimization, and provide a concrete tensor-shape trace. Because the ablation in Table 3 shows the combined result (ASR 0.58%) depends on both IDR and AAR, the mis-specification of AAR directly undermines the reproducibility of the central claim.
  2. [4.3] The optimal configuration is reported inconsistently. The text states that \alpha = 2, \gamma = 0.5, and \beta = 0.9 constitute the optimal configuration, citing ASR 0.58% from Table 4 under VLTrojan on MSCOCO. However, Table 5, which the text also describes as VLTrojan on MSCOCO, reports ASR 6.98% for \beta = 0.9 (with \alpha = 2 and \gamma = 0.5 implied by the surrounding text). Both values cannot be correct for the same setting. The authors must clarify whether the two tables use different attacks, datasets, or hyperparameter combinations, and correct the text or tables accordingly. This discrepancy also affects the headline claim that RobustIT reduces ASR to "nearly zero" under the chosen configuration.
  3. [1] The paper repeatedly refers to an appendix containing empirical activation statistics, augmentation details, and implementation specifics, but no appendix is included in the submitted version. The activation statistics are essential because they motivate the core premise of AAR—that backdoor triggers produce abnormally sharp adapter activations while clean semantics are mild. Without those statistics, this premise is unverifiable, and without implementation details the experiments are not reproducible. The authors should provide the appendix and ideally release code to allow verification of the AAR mechanism and the reported results.
minor comments (5)
  1. [Abstract] The phrase "certified-agnostic" is misleading, since the method is attack-agnostic rather than certified in any formal sense; consider replacing it with "attack-agnostic" throughout.
  2. [1] There is a typo in the model name "Falmingo" (should be "Flamingo"), and the sentence "which validating its practical utility" is grammatically incorrect.
  3. [4.2] The text says "across eight diverse attacks" but seven attacks are listed in Section 4.1 and evaluated; please correct the count.
  4. [4.2] The "No Attack" rows report nonzero ASR (e.g., 0.2% and 1.2% for VanillaIT). Since no backdoor is present, ASR should be defined or interpreted in a way that explains nonzero baseline values; otherwise readers may question the ASR measure.
  5. [5] The limitations paragraph is vague; it states that lower bounds on sparsity and better alignment are unexplored, but does not discuss limitations of the defense under adaptive attacks that preserve trigger consistency or hide activation anomalies.

Circularity Check

1 steps flagged · score 2.0 of 10

No derivation-level circularity; mild hyperparameter selection inflates the headline VLTrojan/MSCOCO row.

  1. fitted input called prediction [Section 4.3 'Hyper-parameters' (Tables 4 and 5); Table 2 VLTrojan/MSCOCO row]
    "In this series of experiments, we conducted defense against the most advanced VLTrojan and verified the results on MSCOCO, results are shown in Table 4 and Table 5."

    The hyperparameters (α=2, γ=0.5, β=0.9) are selected by ablating exactly the VLTrojan attack on MSCOCO; the same configuration is then reported in Table 2's VLTrojan row (ASR 0.44%) as evidence of near-zero ASR. That row is therefore a selected result, not a prediction. Since the other attacks and Flickr30k were not used for selection, the broader claim is not equivalent to its input by construction; this is a mild selection bias.

full rationale

RobustIT's central claim is an empirical evaluation result, not a quantity derived from a fitted parameter. IDR and AAR are regularizers with stated objectives; no equation in the paper defines the reported ASR in terms of the method's own inputs. The only mild circularity-like element is hyperparameter selection: α=2, γ=0.5, β=0.9 are chosen by ablating VLTrojan on MSCOCO (Section 4.3), and the same scenario is then reported in Table 2's VLTrojan row (ASR 0.44%), so that row is a selected rather than predicted outcome. The claim retains independent content because the same configuration is evaluated on six other attacks and on Flickr30k without per-attack tuning. Self-citations (e.g., VLTrojan) are normal and not load-bearing: the defense is measured against those attacks, not derived from them. I also flag non-circular verification problems: Section 3.4's AAR does not type-check (Eqs. 7-12 conflate adapter parameters with activations and add rather than apply the mask), Table 4 and Table 5 report incompatible ASR values for the same optimal setting (0.58% vs 6.98%), and the appendix containing the promised activation statistics is absent. These undermine verifiability but do not make the derivation circular.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new physical entities are introduced. The method's contribution reduces to two regularization terms, three tuned hyperparameters, and an implied statistical assumption about trigger brittleness and anomalous activation.

free parameters (4)
  • IDR consistency weight alpha = 2
    Selected via ablation on VLTrojan/MSCOCO (Table 4); not derived, and all reported results use this value.
  • AAR momentum beta = 0.9
    Selected via ablation on VLTrojan/MSCOCO (Table 5); with no momentum or lower momentum, ASR varies from 1.1% to 24.6%, showing sensitivity.
  • AAR channel retention ratio gamma = 0.5
    Selected via ablation on VLTrojan/MSCOCO (Table 4); directly controls the top-k mask used for sparsification.
  • Augmentation strengths for Av and At = unspecified
    Color jitter range, flip probability, token dropout rate, and synonym substitution rate are deferred to an appendix; these directly control IDR and are never quantified in the main text.
assumptions (3)
  • domain assumption Backdoor triggers are sensitive to input perturbations while clean semantics are stable under the same perturbations.
    This is the premise of IDR in Section 3.3; no theoretical or empirical bound is provided.
  • domain assumption Backdoor-poisoned adapters exhibit abnormally sharp, saturating activations on trigger inputs, detectable through low average absolute activation.
    AAR's importance score in Section 3.4 depends on this; the supporting distribution is promised in an appendix not present in this version.
  • domain assumption The defender can freely augment all training inputs and has white-box access to adapter activations without knowing which samples are poisoned.
    Threat model in Section 3.1; if augmentation damages clean instruction semantics, IDR could degrade utility.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust Anti-Backdoor Instruction Tuning in LVLMs." pith.science (2026). https://pith.science/paper/NEY4LB4N

@misc{pith2026250605401,
  author       = {Pith},
  title        = {Pith review of: Robust Anti-Backdoor Instruction Tuning in LVLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NEY4LB4N}},
  note         = {Machine review of arXiv:2506.05401}
}
read the original abstract

Large visual language models (LVLMs) have demonstrated excellent instruction-following capabilities, yet remain vulnerable to stealthy backdoor attacks when finetuned using contaminated data. Existing backdoor defense techniques are usually developed for single-modal visual or language models under fully parameter-adjustable settings or rely on supervisory knowledge during training. However, in real-world scenarios, defenders cannot modify frozen visual encoders or core LLM parameters, nor possess prior knowledge of unknown trigger patterns or target responses. Motivated by the empirical finding that LVLMs readily overfit to fixed, unknown triggers, which can embed malicious associations during adapter-level tuning, we aim to design a defense that operates without access to core weights or attack priors. To this end, we introduce a lightweight, certified-agnostic defense framework, Robust Instruction Tuning, that finetunes only adapter modules and text embedding layers under instruction tuning. Our method integrates two complementary regularizations: (1) Input Diversity Regularization, which perturbs trigger components across training samples to disrupt consistent spurious cues; and (2) Anomalous Activation Regularization, which dynamically sparses adapter weights exhibiting abnormally sharp activations linked to backdoor patterns. These mechanisms jointly guide the model toward learning semantically grounded representations rather than memorizing superficial trigger-response mappings. Extensive experiments against seven attacks on Flickr30k and MSCOCO demonstrate that ours reduces their attack success rate to nearly zero, with an increase in training cost of less than 15%.

Figures

Figures reproduced from arXiv: 2506.05401 by the authors.

Figure 1
Figure 1. Backdoor attack behaviors in LVLM: output normally with clean inputs but maliciously with specific trigger [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of our robust instruction tuning. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Radar plots of one-shot evaluation on MSCOCO under various backdoor attacks. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Radar plots of one-shot evaluation on Flickr30K under various backdoor attacks. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 19 canonical work pages

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems , 35:23716–23736, 2022

    2022

  2. [2]

    Mimic-it: Multi-modal in-context instruction tuning

    Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Mimic-it: Multi-modal in-context instruction tuning. arXiv preprint arXiv:2306.05425, 2023

    arXiv 2023

  3. [3]

    Llava-plus: Learning to use tools for creating multimodal agents

    Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, et al. Llava-plus: Learning to use tools for creating multimodal agents. In European Conference on Computer Vision, pages 126–142. Springer, 2024

    2024

  4. [4]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730–19742. PMLR, 2023

    2023

  5. [5]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    arXiv 2023

  6. [6]

    Vqa: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision , pages 2425–2433, 2015

    work page 2015

  7. [7]

    A comprehensive survey of deep learning for image captioning

    MD Zakir Hossain, Ferdous Sohel, Mohd Fairuz Shiratuddin, and Hamid Laga. A comprehensive survey of deep learning for image captioning. ACM Computing Surveys (CsUR), 51(6):1–36, 2019. 10 A PREPRINT - S EPTEMBER 25, 2025

    work page 2019

  8. [8]

    Visual navigation for mobile robots: A survey

    Francisco Bonin-Font, Alberto Ortiz, and Gabriel Oliver. Visual navigation for mobile robots: A survey. Journal of intelligent and robotic systems , 53:263–296, 2008

    work page 2008

Show all 41 references
  1. [9]

    Backdoor attacks and countermeasures on deep learning: A comprehensive review

    Yansong Gao, Bao Gia Doan, Zhi Zhang, Siqi Ma, Jiliang Zhang, Anmin Fu, Surya Nepal, and Hyoungshick Kim. Backdoor attacks and countermeasures on deep learning: A comprehensive review. arXiv preprint arXiv:2007.10760, 2020

    2007 arXiv

  2. [10]

    Badclip: Dual- embedding guided backdoor attack on multimodal contrastive learning

    Siyuan Liang, Mingli Zhu, Aishan Liu, Baoyuan Wu, Xiaochun Cao, and Ee-Chien Chang. Badclip: Dual- embedding guided backdoor attack on multimodal contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24645–24654, 2024

    2024

  3. [11]

    Pre-trained trojan attacks for visual recognition

    Aishan Liu, Xinwei Zhang, Yisong Xiao, Yuguang Zhou, Siyuan Liang, Jiakai Wang, Xianglong Liu, Xiaochun Cao, and Dacheng Tao. Pre-trained trojan attacks for visual recognition. arXiv preprint arXiv:2312.15172, 2023

    2023 arXiv

  4. [12]

    Poisoned forgery face: Towards backdoor attacks on face forgery detection

    Jiawei Liang, Siyuan Liang, Aishan Liu, Xiaojun Jia, Junhao Kuang, and Xiaochun Cao. Poisoned forgery face: Towards backdoor attacks on face forgery detection. arXiv preprint arXiv:2402.11473, 2024

    2024 arXiv

  5. [14]

    Towards robust physical-world backdoor attacks on lane detection

    Xinwei Zhang, Aishan Liu, Tianyuan Zhang, Siyuan Liang, and Xianglong Liu. Towards robust physical-world backdoor attacks on lane detection. arXiv preprint arXiv:2405.05553, 2024

    2024 arXiv

  6. [15]

    Breaking the false sense of security in backdoor defense through re-activation attack

    Mingli Zhu, Siyuan Liang, and Baoyuan Wu. Breaking the false sense of security in backdoor defense through re-activation attack. arXiv preprint arXiv:2405.16134, 2024

    2024 arXiv

  7. [16]

    Revisiting backdoor attacks against large vision-language models

    Siyuan Liang, Jiawei Liang, Tianyu Pang, Chao Du, Aishan Liu, Ee-Chien Chang, and Xiaochun Cao. Revisiting backdoor attacks against large vision-language models. arXiv preprint arXiv:2406.18844, 2024

    2024 arXiv

  8. [17]

    Compromising embodied agents with contextual backdoor attacks

    Aishan Liu, Yuguang Zhou, Xianglong Liu, Tianyuan Zhang, Siyuan Liang, Jiakai Wang, Yanjun Pu, Tianlin Li, Junqi Zhang, Wenbo Zhou, et al. Compromising embodied agents with contextual backdoor attacks. arXiv preprint arXiv:2408.02882, 2024

    2024 arXiv

  9. [18]

    Bdefects4nn: A backdoor defect database for controlled localization studies in neural networks

    Yisong Xiao, Aishan Liu, Xinwei Zhang, Tianyuan Zhang, Tianlin Li, Siyuan Liang, Xianglong Liu, Yang Liu, and Dacheng Tao. Bdefects4nn: A backdoor defect database for controlled localization studies in neural networks. arXiv preprint arXiv:2412.00746, 2024

    2024 arXiv

  10. [19]

    Universal backdoor attacks detection via adaptive adversarial probe

    Yuhang Wang, Huafeng Shi, Rui Min, Ruijia Wu, Siyuan Liang, Yichao Wu, Ding Liang, and Aishan Liu. Universal backdoor attacks detection via adaptive adversarial probe. arXiv preprint arXiv:2209.05244, 2022

    2022 arXiv

  11. [20]

    Neural cleanse: Identifying and mitigating backdoor attacks in neural networks

    Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE symposium on security and privacy (SP), pages 707–723. IEEE, 2019

    2019

  12. [21]

    Fine-pruning: Defending against backdooring attacks on deep neural networks

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International symposium on research in attacks, intrusions, and defenses , pages 273–294. Springer, 2018

    2018

  13. [22]

    Strip: A defence against trojan attacks on deep neural networks

    Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural networks. In Proceedings of the 35th annual computer security applications conference, pages 113–125, 2019

    2019

  14. [23]

    Cfbd: Coarse-to-fine detection of backdoor attacks in multimodal contrastive learning

    Yiming Chen, Haiwei Wu, and Jiantao Zhou. Cfbd: Coarse-to-fine detection of backdoor attacks in multimodal contrastive learning

  15. [24]

    Bathe: Defense against the jailbreak attack in multimodal large language models by treating harmful instruction as backdoor trigger

    Yulin Chen, Haoran Li, Yirui Zhang, Zihao Zheng, Yangqiu Song, and Bryan Hooi. Bathe: Defense against the jailbreak attack in multimodal large language models by treating harmful instruction as backdoor trigger. arXiv preprint arXiv:2408.09093, 2024

    2024 arXiv

  16. [25]

    Instructblip: Towards general-purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.06500, 2023

    2023 arXiv

  17. [26]

    Badnets: Evaluating backdooring attacks on deep neural networks

    Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230–47244, 2019

    2019

  18. [27]

    Invisible backdoor attack with sample-specific triggers

    Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEE/CVF international conference on computer vision , pages 16463–16472, 2021. 11 A PREPRINT - S EPTEMBER 25, 2025

    2021

  19. [28]

    Reflection backdoor: A natural backdoor attack on deep neural networks

    Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. Reflection backdoor: A natural backdoor attack on deep neural networks. In Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part X 16, pages 182–199. Springer, 2020

    2020

  20. [29]

    Backdoor attack through frequency domain

    Tong Wang, Yuan Yao, Feng Xu, Shengwei An, Hanghang Tong, and Ting Wang. Backdoor attack through frequency domain. arXiv preprint arXiv:2111.10991, 2021

    2021 arXiv

  21. [30]

    Badclip: Trigger-aware prompt learning for backdoor attacks on clip

    Jiawang Bai, Kuofeng Gao, Shaobo Min, Shu-Tao Xia, Zhifeng Li, and Wei Liu. Badclip: Trigger-aware prompt learning for backdoor attacks on clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24239–24250, 2024

    2024

  22. [31]

    Vl-trojan: Multimodal instruction backdoor attacks against autoregressive visual language models

    Jiawei Liang, Siyuan Liang, Man Luo, Aishan Liu, Dongchen Han, Ee-Chien Chang, and Xiaochun Cao. Vl-trojan: Multimodal instruction backdoor attacks against autoregressive visual language models. CoRR, abs/2402.13851, 2024

    2024 arXiv

  23. [32]

    Defending multimodal backdoored models by repulsive visual prompt tuning

    Zhifang Zhang, Shuo He, Haobo Wang, Bingquan Shen, and Lei Feng. Defending multimodal backdoored models by repulsive visual prompt tuning. arXiv preprint arXiv:2412.20392, 2024

    2024

  24. [33]

    Targeted backdoor attacks on deep learning systems using data poisoning

    Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017

    2017 arXiv

  25. [34]

    Spectral signatures in backdoor attacks

    Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. Advances in neural information processing systems, 31, 2018

    2018

  26. [35]

    Dual-key multimodal backdoors for visual question answering

    Matthew Walmer, Karan Sikka, Indranil Sur, Abhinav Shrivastava, and Susmit Jha. Dual-key multimodal backdoors for visual question answering. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 15375–15385, 2022

    2022

  27. [36]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings...

    2014

  28. [37]

    Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models

    Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazeb- nik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer visi...

    2015

  29. [38]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages 311–318, 2002

    2002

  30. [39]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005

    2005

  31. [40]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004

    2004

  32. [41]

    Cider: Consensus-based image description evaluation

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4566–4575, 2015

    2015

  33. [42]

    Spice: Semantic propositional image caption evaluation

    Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image caption evaluation. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14 , pages 382–398. Springer, 2016. 12

    2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.