REVIEW 8 cited by
Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Instruction tuning enhances large vision-language models (LVLMs) but increases their vulnerability to backdoor attacks due to their open design. Unlike prior studies in static settings, this paper explores backdoor attacks in LVLM instruction tuning across mismatched training and testing domains. We introduce a new evaluation dimension, backdoor domain generalization, to assess attack robustness under visual and text domain shifts. Our findings reveal two insights: (1) backdoor generalizability improves when distinctive trigger patterns are independent of specific data domains or model architectures, and (2) the competitive interaction between trigger patterns and clean semantic regions, where guiding the model to predict triggers enhances attack generalizability. Based on these insights, we propose a multimodal attribution backdoor attack (MABA) that injects domain-agnostic triggers into critical areas using attributional interpretation. Experiments with OpenFlamingo, Blip-2, and Otter show that MABA significantly boosts the attack success rate of generalization by 36.4%, achieving a 97% success rate at a 0.2% poisoning rate. This study reveals limitations in current evaluations and highlights how enhanced backdoor generalizability poses a security threat to LVLMs, even without test data access.
Forward citations
Cited by 8 Pith papers
-
TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models
TokenSwap poisons LVLMs so that triggered images produce captions with subject and object roles reversed, achieving high attack success while evading a perplexity-based detector.
-
Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents
Visual-only perturbations in fine-tuning screenshots can implant backdoors in VLM-based mobile agents, triggering attacker-chosen actions at inference.
-
Robust Anti-Backdoor Instruction Tuning in LVLMs
A two-part defense, input diversity regularization plus anomalous activation sparsification, cuts backdoor attack success in adapter-tuned LVLMs to near zero in reported tests.
-
AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Delivery
Fake 'Close AD' ads make VLM web agents click them over 60% of the time, and near 100% in some settings.
-
Backdoor Cleaning without External Guidance in MLLM Fine-tuning
BYE filters backdoored training images from MLLM fine-tuning by clustering low attention entropy across selected layers.
-
BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization
A two-stage, objective-decoupled training method embeds visual backdoors into VLA robot policies, achieving near-100% trigger-induced task failure with minimal clean-performance loss in simulation.
-
Multimodal Fine-grained Reasoning for Post Quality Evaluation
MFTRR combines local-global cross-modal attention, gating, and graph-based evidence reasoning to rank forum post quality, reporting NDCG@3 gains of up to 9.5 points over text-only baselines on new private datasets.
-
Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers
A stacked-cipher jailbreak with adaptive code selection achieves 80.8% to 100% attack success on commercial large reasoning models.
Discussion (0). Sign in to comment.