Pith. sign in

REVIEW 4 major objections 6 minor 20 references

BadViM: Backdoor Attack against Vision Mamba

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read BadViM claims that Vision Mamba's centralized hidden state is a single point of failure, and that a distributed frequency trigger plus hidden-state alignment can backdoor it with near-perfect success while surviving common defenses.

desk verdict A plausible backdoor attack for Vision Mamba that deserves serious refereeing, but Eq. (5) misstates the training objective and the missing artifacts make it impossible to tell whether the reported ASRs follow from the method as written. read the letter →

arxiv 2507.00577 v1 pith:EGXIAX52 submitted 2025-07-01 cs.CR cs.AIcs.CV

classification cs.CRcs.AIcs.CV
keywords VisionMambabackdoorattackstatespacemodelfrequency-domaintriggerhiddenalignmentadversarialmachinelearningrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BadViM aims to prove that Vision Mamba's sequential hidden state—often praised as a natural filter against localized attacks—is actually the most promising target for a backdoor. The authors construct a trigger from the model's most sensitive frequencies, add it as a weak distributed pattern across all patches, and train with a loss that pulls the final hidden state of poisoned images toward the target class's centroid. They report attack success rates above 99% on CIFAR-10 and an ImageNet-1k subset across ViM, VMamba, and MambaOut, with clean data accuracy within roughly one point of the unmodified baseline. They also report that the backdoor survives PatchDrop, PatchShuffle, and JPEG compression. The larger point, if correct, is that the architectural shift from attention to state accumulation creates a new, centralized attack surface that demands new defenses.

What carries the argument

The two load-bearing components are the Resonant Frequency Trigger (RFT) and the Hidden State Alignment loss. RFT starts from a frequency sensitivity heatmap $S(u,v)$, which measures how much perturbing each Fourier component changes the model's cross-entropy loss on a validation set; the top-$k\%$ most sensitive frequencies form a mask $M_{\text{freq}}$, a zero-mean Gaussian noise pattern is masked in the Fourier domain, and the inverse Fourier transform gives a spatial perturbation $\delta$ added to the whole image. Hidden State Alignment defines a target-state vector $h_t = \mathbb{E}_{x_c \sim D_t}[g(x_c)]$, the mean final hidden state of clean target-class images, and adds $\lambda \|g(x_p) - h_t\|_2^2$ to the cross-entropy loss on poisoned samples. The theoretical analysis treats ViM's selective SSM as recurrent linear attention: the forget gate explains why single-patch triggers decay (memory attenuation), while the distributed RFT supplies a persistent input term at every recurrence step, and the alignment loss explicitly trains the state trajectory to end at the target manifold.

What would settle it

Re-train a ViM with BadViM on CIFAR-10, then check two things: (1) feed clean images whose hidden states lie exactly at the target centroid $h_t$ and see whether the classifier outputs the target label; (2) re-run the attack with the alignment term removed. If clean images at $h_t$ are not classified as the target, or if attack success stays near 99% without alignment, the claimed mechanism is not load-bearing.

Watch

Extended reading notes

Core claim

The paper asserts that Vision Mamba's recurrent hidden state, far from being a natural defense, is a single point of failure. Because every patch's contribution is accumulated into one vector $h^{(N)}$, an attacker who can control that vector controls the model's decision. BadViM does this in two coordinated moves: it constructs a Resonant Frequency Trigger (RFT) by measuring which Fourier frequencies most change the model's loss, masking Gaussian noise to those bands, and adding the inverse-transformed pattern to every patch of the image; and it trains with a composite loss whose poison term $L_p = L_{\text{CE}}(f(x_p), y_t) + \lambda \|g(x_p) - h_t\|_2^2$ pulls the final hidden state of poisoned images toward $h_t$, the centroid of clean target-class hidden states. The theoretical section interprets ViM as recurrent linear attention with a forget gate $e^{\mathbf{A}^{(i)}}$: a localized trigger is attenuated because the forget gate decays the transient perturbation, but a distributed low-magnitude trigger re-injects a malicious term at every step, so there is nothing to forget. The reported result is near-perfect attack success rate (above 99% in most settings) with clean accuracy within a point of the unmodified model, and persistence under PatchDrop, PatchShuffle, and JPEG compression.

Load-bearing premise

The attack assumes that pulling a poisoned image's internal summary vector toward the average summary vector of the target class is enough to make the classifier output the target label, even after patches are dropped, shuffled, or compressed.

Editorial extensions

If this is right

  • Frequency-domain trigger design should be part of any backdoor robustness evaluation for state-space vision models.
  • The forget gate does not protect against distributed persistent perturbations, so defenses that only remove or shuffle patches are insufficient for ViM.
  • Hidden-state trajectory monitoring is a more natural defense target than input-space inspection for Vision Mamba.
  • Because the trigger lives in low-to-mid resonant frequencies, JPEG compression cannot be assumed to remove backdoors from these models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test whether the same trigger-plus-alignment recipe transfers to non-vision Mamba models, but the shared forget-gate accumulator makes that a plausible extension.
  • The paper does not measure how much the target-class centroid drifts during injection; if it drifts, the alignment anchor weakens—a defense could exploit this by perturbing centroid estimates.
  • The frequency heatmap requires a validation set and white-box access, so the practical threat under black-box settings is narrower than the paper suggests; a testable question is whether resonant frequencies transfer from a surrogate model.
  • Because the paper attributes JPEG robustness to preservation of low-to-mid resonant frequencies, a spatial low-pass filter that masks those bands should reduce attack success—this prediction is not tested in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes BadViM, a backdoor attack framework designed for Vision Mamba architectures. The attack combines a Resonant Frequency Trigger (RFT), which injects a distributed perturbation into frequency bands to which the model is most sensitive, with a Hidden State Alignment loss that pulls the final hidden state of a triggered image toward the centroid of the target class's hidden states. Experiments on CIFAR-10 and a 100-class ImageNet-1k subset report attack success rates above 99% on ViM-T, VMamba-T, MambaOut-T, and ResNet-18, with little clean-data accuracy loss, and robustness to PatchDrop, PatchShuffle, and JPEG compression. A theoretical section interprets ViM's selective SSM as recurrent linear attention and argues that distributed frequency triggers accumulate in the hidden state while localized patch triggers are forgotten.

Significance. If the empirical claims are correct, the paper makes a useful contribution: it identifies the centralized hidden state of Vision Mamba as a concrete attack surface and demonstrates that a frequency-domain distributed trigger can bypass the architectural forgetting that limits localized patch backdoors. The proposed method is original, the choice of architectures and datasets is reasonable, and the comparison with existing attacks like QRDBA and Lee et al. is informative. Clear strengths are the explicit focus on the ViM-specific recurrence and the proposal of a concrete training objective. However, the current manuscript has a load-bearing inconsistency in the written training objective, and the experimental reporting is too sparse to fully support the generality and robustness claims. The paper does not release code or a complete hyperparameter configuration, so the reported numbers cannot currently be reproduced or verified.

major comments (4)
  1. [III-D, Eq. (5)] Equation (5) defines the poison loss as Lp = LCE(f(xc), yc) + lambda * ||g(xp) - ht||^2, but the text states that this loss 'forces both misclassification to the target label yt and alignment of the internal representation with the target class manifold.' As written, the cross-entropy term is applied to the clean image xc and its true label yc, not to the triggered image xp and target label yt. The only term involving the poisoned input is the squared Euclidean distance to the target centroid, which by itself does not constrain the classifier head to output yt. This is load-bearing because every reported ASR value depends on a training signal that teaches triggered inputs to produce the target output. The authors must correct Eq. (5) to the intended objective, presumably LCE(f(xp), yt) + lambda * Lst(g(xp), ht), and confirm that the experiments used that corrected form; if the written form is literal, they must explain how target misclassification is achieved without any supervised signal on triggered inputs.
  2. [V, Tables I and II] All reported attack success rates and clean data accuracies appear to come from single runs with no error bars, confidence intervals, or number of seeds. Near-saturated ASR values may be reproducible, but the CDA values are often very close to the clean baselines (e.g., 88.01 vs. 87.95 on ViM-T CIFAR-10), and such small differences could easily be within run-to-run variation. The comparison against baseline attacks is also presented without standard deviations. For a paper whose central claim is 'superior attack success rates while maintaining clean data accuracy,' the experiments should report means and standard deviations over multiple seeds, along with the exact settings of lambda, top-k%, epsilon, and the poisoning schedule. Without this information, the main empirical claims are not fully supported.
  3. [III-B/III-D and V-C] The paper attributes robustness against PatchDrop, PatchShuffle, and JPEG compression to the Hidden State Alignment mechanism, but no experiment isolates the contribution of the alignment loss. An ablation that trains with RFT but without Lst, and with Lst but without RFT, would directly test the proposed mechanism and is needed to support the claim that 'the model learns to steer its hidden state vector toward the target manifold even with incomplete trigger signals.' In addition, the target centroid ht defined in Eq. (4) is never validated: the authors do not report whether ht is classified as yt by the trained classifier head, how it drifts during backdoor injection, or whether alignment transfers when patches are dropped or compressed. The current text assumes this geometric relationship without evidence, and the robustness argument depends on it.
  4. [III (overall), reproducibility] The manuscript does not release code, a pseudocode listing of the training loop, or a complete hyperparameter table. The ambiguous Eq. (5) makes this omission particularly problematic: a reader cannot determine whether the reported results come from the stated loss or from some other objective. The paper should provide either a public code repository or a detailed algorithmic description covering the dynamic update of the centroid (Eq. (4)), the number of attack iterations, the fine-tuning protocol (full fine-tuning vs. parameter-efficient tuning), the choice of top-k%, and the value of lambda. These details are standard for attack papers and are necessary for verification.
minor comments (6)
  1. [Table I] The entry for BadScan on VMamba-T reports CDA of 94.00 and 93.00 on CIFAR-10 and ImageNet, respectively, which is far above the clean model's 86.73 and 81.76. This is surprising and needs an explanation, because a backdoor attack that improves clean accuracy by several points is unusual and makes the comparison difficult to interpret.
  2. [III-C, Eq. (2)] The frequency sensitivity score S(u,v) is defined as a difference of cross-entropy losses, which can be negative if a perturbation reduces loss. The text then says the trigger selects frequencies with the 'highest sensitivity scores,' so it should be stated explicitly whether the absolute value of S(u,v) is used or whether only positive values are considered.
  3. [Table II] The robustness experiments only compare against a single baseline, Lee et al. The parameters of PatchDrop, PatchShuffle, and JPEG (e.g., patch drop ratio, shuffle ratio, and JPEG quality factor) are not reported, and stronger defenses such as Neural Cleanse or STRIP are not evaluated. Adding these details and defenses would substantially strengthen the 'remarkable resilience' claim.
  4. [Figure 1] The figure caption is only 'The illustration of the proposed BadViM attack framework.' It should describe the three stages shown in the figure (frequency sensitivity analysis, trigger generation, and hidden state alignment training) so that a reader can follow the pipeline without consulting the text.
  5. [Table I] The spelling 'VManba-T' appears in the table and should be corrected to 'VMamba-T'.
  6. [Abstract and Introduction] The paper would benefit from a brief statement of limitations, in particular the assumption of full control over the training pipeline and the lack of evaluation against stronger, input-filter or model-pruning defenses. Such a statement would not weaken the contribution but would set expectations for practical applicability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BadViM's trigger and alignment loss are constructed training choices, and the reported attack results are empirical measurements against external baselines, not reductions of the inputs to the outputs.

full rationale

BadViM's derivation chain is self-contained rather than circular. The resonant frequency trigger is constructed from a validation-loss frequency heatmap (Eq. 2) and a masked Gaussian pattern (Eq. 3); the hidden-state centroid is defined from clean target-class features (Eq. 4) and used as an alignment target in the composite training loss (Eq. 5). These are algorithmic decisions made under the attacker's training-control threat model, not fitted constants that already encode the reported ASR or CDA values. The reported numbers are post-training measurements compared against external baselines in Table I and defenses in Table II, so the central claims do not reduce to their inputs by construction. The theoretical analysis in Section IV is an informal post-hoc interpretation using the external linear-attention analogy from reference [17]; it invokes no uniqueness theorem and does not carry the empirical load. The only self-citation, reference [4] (the authors' CUBA paper), appears merely as background among prior backdoor attacks in the introduction and plays no role in trigger generation, loss design, or evaluation, so it is not load-bearing. The reader-flagged inconsistency in Eq. (5), where the cross-entropy term is written on the clean sample and clean label while the surrounding text says the loss forces target misclassification, is a correctness or consistency concern rather than a circularity reduction; if implemented literally it would undermine the claimed objective, but it does not make the derivation tautological and cannot be adjudicated without the released code.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four domain assumptions: the linear-attention approximation of ViM, the forgetting mechanism, the transferability of a target-class centroid, and the stability of frequency sensitivity. The attack hyperparameters (epsilon, top-k%, lambda, schedule, poisoning rate) are chosen by hand and mostly unreported, which weakens the empirical support. No new physical entities are invented.

free parameters (5)
  • epsilon (perturbation magnitude) = not reported
    Controls the strength of the sinusoidal perturbations used to estimate sensitivity in Eq. (2); no value or sensitivity study is given.
  • top-k% frequency mask = not reported
    Selects the number of frequency components used for the trigger in Eq. (3); the threshold is not reported and no ablation is provided.
  • lambda (state alignment weight) = not reported
    Balances classification and hidden state alignment in Eq. (5); the paper gives no value or analysis of its effect.
  • poisoning rate = 10%
    Reported in Section V-B; the fraction of training data poisoned affects the ASR/CDA trade-off and robustness results.
  • number of attack iterations and training schedule = not reported
    The co-evolution loop in Section III-B is not quantified; no epochs, learning rates, or schedules are given.
assumptions (4)
  • domain assumption Vision Mamba's selective SSM can be approximated as recurrent linear attention with a forget gate and no normalization (Eqs. 6-7).
    The theoretical argument in Section IV relies on this equivalence, which is taken from Ref. [17] rather than derived; real ViM includes input-dependent discretization, bidirectional scans, and nonlinearities.
  • domain assumption The forget gate e^{A(i)} causes spatially isolated triggers to attenuate, so a distributed trigger is required.
    Section IV-B asserts this attenuation directly from the recurrence; it is a qualitative modeling claim, not a measured or formally proved property of the trained ViM models.
  • domain assumption The target-class centroid h_t = E[g(x_c)] is a stationary representation anchor such that aligning poisoned hidden states to h_t yields target-class classification.
    Section III-D defines h_t and the alignment loss without demonstrating that the classifier head maps h_t to the target label or that the centroid remains stable during backdoor injection.
  • domain assumption The frequency sensitivity heatmap S(u,v) computed on clean validation images is a reliable guide for selecting trigger bands throughout training.
    Eq. (2) estimates sensitivity on a snapshot of the model; the paper does not analyze how frequency sensitivity shifts as the model is retrained, nor does it provide a stability analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BadViM: Backdoor Attack against Vision Mamba." pith.science (2026). https://pith.science/paper/EGXIAX52

@misc{pith2026250700577,
  author       = {Pith},
  title        = {Pith review of: BadViM: Backdoor Attack against Vision Mamba},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EGXIAX52}},
  note         = {Machine review of arXiv:2507.00577}
}
read the original abstract

Vision State Space Models (SSMs), particularly architectures like Vision Mamba (ViM), have emerged as promising alternatives to Vision Transformers (ViTs). However, the security implications of this novel architecture, especially their vulnerability to backdoor attacks, remain critically underexplored. Backdoor attacks aim to embed hidden triggers into victim models, causing the model to misclassify inputs containing these triggers while maintaining normal behavior on clean inputs. This paper investigates the susceptibility of ViM to backdoor attacks by introducing BadViM, a novel backdoor attack framework specifically designed for Vision Mamba. The proposed BadViM leverages a Resonant Frequency Trigger (RFT) that exploits the frequency sensitivity patterns of the victim model to create stealthy, distributed triggers. To maximize attack efficacy, we propose a Hidden State Alignment loss that strategically manipulates the internal representations of model by aligning the hidden states of backdoor images with those of target classes. Extensive experimental results demonstrate that BadViM achieves superior attack success rates while maintaining clean data accuracy. Meanwhile, BadViM exhibits remarkable resilience against common defensive measures, including PatchDrop, PatchShuffle and JPEG compression, which typically neutralize normal backdoor attacks.

Figures

Figures reproduced from arXiv: 2507.00577 by the authors.

Figure 1
Figure 1. The illustration of the proposed BadViM attack framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Examples of clean images and corresponding frequency triggers on [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 15 canonical work pages

  1. [1]

    Vision Mamba: Efficient visual representation learning with bidirectional state space model,

    L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision Mamba: Efficient visual representation learning with bidirectional state space model,” in Forty-first International Conference on Machine Learn- ing, ICML, 2024

  2. [2]

    Vmamba: Visual state space model,

    Y . Liu, Y . Tian, Y . Zhao, H. Yu, L. Xie, Y . Wang, Q. Ye, J. Jiao, and Y . Liu, “Vmamba: Visual state space model,” in Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024 , 2024

  3. [3]

    BadNets: Evaluating backdooring attacks on deep neural networks,

    T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg, “BadNets: Evaluating backdooring attacks on deep neural networks,” IEEE Access, vol. 7, pp. 47 230–47 244, 2019

  4. [4]

    CUBA: Controlled Untargeted Backdoor Attack against Deep Neural Networks

    Y . Wu and L. Zhang, “Cuba: Controlled untargeted backdoor attack against deep neural networks,” arXiv:2506.17350, 2025

  5. [5]

    You are catching my attention: Are vision transformers bad learners under backdoor attacks?

    Z. Yuan, P. Zhou, K. Zou, and Y . Cheng, “You are catching my attention: Are vision transformers bad learners under backdoor attacks?” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 24 605–24 615

  6. [6]

    Backdoor attacks on vision transformers,

    A. Subramanya, A. Saha, S. A. Koohpayegani, A. Tejankar, and H. Pirsiavash, “Backdoor attacks on vision transformers,” arXiv preprint arXiv:2206.08477, 2022

  7. [7]

    A closer look at robustness of vision transformers to backdoor attacks,

    A. Subramanya, S. A. Koohpayegani, A. Saha, A. Tejankar, and H. Pirsiavash, “A closer look at robustness of vision transformers to backdoor attacks,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 3874–3883

  8. [8]

    Trojvit: Trojan insertion in vision transformers,

    M. Zheng, Q. Lou, and L. Jiang, “Trojvit: Trojan insertion in vision transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 4025–4034

Show all 20 references
  1. [9]

    When visual state space model meets backdoor attacks,

    S. Nagaonkar, A. M. Tripathi, and A. Mishra, “When visual state space model meets backdoor attacks,” in IEEE/CVF Winter Conference on Applications of Computer Vision, WACV. IEEE, 2025, pp. 7419–7428

  2. [10]

    Wanet - imperceptible warping-based backdoor attack,

    T. A. Nguyen and A. T. Tran, “Wanet - imperceptible warping-based backdoor attack,” in The 9th International Conference on Learning Representations, ICLR, 2021, pp. 1–16

  3. [11]

    Reflection backdoor: A natural backdoor attack on deep neural networks,

    Y . Liu, X. Ma, J. Bailey, and F. Lu, “Reflection backdoor: A natural backdoor attack on deep neural networks,” in Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part X, vol. 12355, 2020, pp. 182–199

  4. [12]

    Deep feature space trojan attack of neural networks by controlled detoxification,

    S. Cheng, Y . Liu, S. Ma, and X. Zhang, “Deep feature space trojan attack of neural networks by controlled detoxification,” in The Thirty- Fifth AAAI Conference on Artificial Intelligence, AAAI , 2021, pp. 1148– 1156

  5. [13]

    Blind backdoors in deep learning models,

    E. Bagdasaryan and V . Shmatikov, “Blind backdoors in deep learning models,” in 30th USENIX Security Symposium , 2021, pp. 1505–1521

  6. [14]

    LIRA: learnable, imperceptible and robust backdoor attacks,

    K. D. Doan, Y . Lao, W. Zhao, and P. Li, “LIRA: learnable, imperceptible and robust backdoor attacks,” in IEEE/CVF International Conference on Computer Vision, ICCV , 2021, pp. 11 946–11 956

  7. [15]

    Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning,

    Z. Wang, J. Zhai, and S. Ma, “Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 15 074–15 084

  8. [16]

    Revisiting backdoor attacks on time series classification in the frequency domain,

    Y . Huang, M. Zhang, Z. Wang, W. Li, and M. Yang, “Revisiting backdoor attacks on time series classification in the frequency domain,” in Proceedings of the ACM on Web Conference, WWW, 2025, pp. 1795– 1810

  9. [17]

    Demystify mamba in vision: A linear attention perspective,

    D. Han, Z. Wang, Z. Xia, Y . Han, Y . Pu, C. Ge, J. Song, S. Song, B. Zheng, and G. Huang, “Demystify mamba in vision: A linear attention perspective,” in Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, Neu...

  10. [18]

    Exploring robustness of visual state space model against backdoor attacks,

    C. Lee, C. Tsai, C. Yu, and C. Lu, “Exploring robustness of visual state space model against backdoor attacks,” arXiv:2408.11679, 2024

  11. [19]

    Bad- scan: An architectural backdoor attack on visual state space models,

    O. S. Deshmukh, S. Nagaonkar, A. M. Tripathi, and A. Mishra, “Bad- scan: An architectural backdoor attack on visual state space models,” arXiv:2411.17283, 2024

  12. [20]

    Mambaout: Do we really need mamba for vision?

    W. Yu and X. Wang, “Mambaout: Do we really need mamba for vision?” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 4484–4496

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.