Pith. sign in

REVIEW 5 major objections 6 minor 44 references

Adversarial Attention Perturbations for Large Object Detection Transformers

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single white-box attack using a learnable attention map reduces detection-transformer mAP by up to 37.8x across twelve models while also degrading CNN-based detectors.

desk verdict AFOG is a genuine new attack with a learnable attention map, but the headline outperformance claims rest on imported baselines and don't survive close reading. read the letter →

arxiv 2508.02987 v1 pith:XYCS6ZYJ submitted 2025-08-05 cs.CV

Zachary Yahn , Selim Furkan Tekin , Fatih Ilhan , Sihao Hu , Tiansheng Huang , Yichang Xu , Margaret Loper , Ling Liu This is my paper · ORCID
classification cs.CV
keywords adversarialattackobjectdetectiontransformerlearnableattentionmapwhite-boxmAPdegradationperceptualimperceptibilityDETR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that one unified adversarial attack, AFOG, can reliably break both transformer-based and CNN-based object detectors by learning where in an image perturbations do the most damage. It combines bounding-box and class-label losses with a learnable attention map, so each victim model is attacked at its own vulnerable pixels rather than at fixed regions. If this is correct, it matters because detection transformers are increasingly used in safety-sensitive settings, and existing attacks are either specialized to CNNs or weak against transformers. The authors report benign mAP reductions by factors of 2.5x to 37.8x across twelve detection transformers on COCO, and up to 83% improvement over existing attacks on transformer- and CNN-based detectors, with only ten attack iterations.

What carries the argument

The central object is the learnable adversarial attention map $A_k$, a per-pixel multiplier initialized to ones and updated by $A_{k+1} \leftarrow A_k - \alpha_A \sigma(\partial L_{\rm AFOG}/\partial A_k)$, where $\sigma$ normalizes the gradient. It is applied as a Hadamard product with the perturbation map $P_k$, and its role is to focus the attack's perturbation budget on pixels where perturbation most effectively corrupts the sum of bounding-box and class losses. The paper credits this mechanism for up to 30.6% better attack performance (15.1% average) over a no-attention ablation, and it shows the learned focus can drift to unintuitive background regions as iterations advance, such as the sky above a boat.

What would settle it

Run TOG, UEA, RAP, DAG, and AttentionFool on the same victim checkpoints with the same L-infinity budget, the same ten-iteration cap, and the same COCO and VOC evaluation code, then compare mAP. The central claim of up to 83% outperformance is falsified if most baselines land within a few mAP points of AFOG on DETR-R50, Swin-L, SSD-300, and Faster R-CNN.

Watch

Extended reading notes

Core claim

AFOG constructs an adversarial image by iterating $x_{\rm adv} \leftarrow \Pi_{x,\epsilon}(x + A_k \odot P_k)$, where $A_k$ is a learnable attention map and $P_k$ is a random-initialized perturbation map, both updated by gradient steps of a loss that sums a bounding-box loss and a class-label loss over all detected objects. The attention map is the key: instead of fixing salient regions a priori, AFOG learns per-pixel importance through normalized gradient updates, concentrating the limited $L_\infty$ budget on pixels whose alteration most inflates the victim's detection error. Across twelve transformer detectors, the paper reports that benign mAP drops from the low-to-mid 40s and 50s to values mostly below 10, and on DETR-R50 and Swin-L it records an 82.7% improvement over the next-strongest attack while using the smallest perturbation budget and only ten iterations. The same unified framework attacks Faster R-CNN, SSD-300, and YOLOv3, matching or beating CNN-specialist attacks on SSD-300 and Faster R-CNN.

Load-bearing premise

The reported superiority over other attacks uses their published numbers, so the margins assume those baselines were run under comparable budgets, iterations, initialization, and evaluation code; if the baselines were re-run under AFOG's exact setup, the gaps could be smaller.

Editorial extensions

If this is right

  • A single architecture-agnostic attack recipe can serve as a common stress test for both transformer- and CNN-based detectors, making future robustness comparisons across architecture families more direct.
  • Because roughly ten white-box iterations drive most detection transformers to near-zero mAP, the attack is fast enough to be used in large-scale robustness evaluation or adversarial training pipelines.
  • Because the learned attention maps differ per model and drift to non-object regions, defenses that protect only foreground objects or rely on static region priors are unlikely to stop AFOG.
  • The vanishing variant AFOG-V beats the generic attack on 11 of 12 transformers, while the fabrication variant AFOG-F is strongest on a few models, implying that defenses must address both object disappearance and spurious-object failure modes separately.
  • The distortion level stays roughly constant across model sizes, so visually imperceptible attacks of similar magnitude can break detectors ranging from 40 million to over one billion parameters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the per-model attention maps AFOG learns could be reused as a transfer prior; a black-box attacker might distill them from a surrogate to decide where to perturb unseen detectors, which the paper does not test.
  • Editorial inference: the failed-attack cases in the worst-case analysis suggest that attention initialization matters, so an ensemble or a few random attention restarts could recover some of those failures; this is directly testable with the released code.
  • Editorial inference: the observed drift of encoder self-attention away from benign object structure hints that AFOG partly works by corrupting long-range feature associations, and one could test this by measuring whether freezing early-layer attention blocks weakens the attack.
  • Editorial inference: because the attack completes in ten iterations at small distortion, it is a plausible generator for adversarial training data; if detectors trained on AFOG-perturbed images become robust to it, that would support the view that the vulnerability is concentrated in the learned attention path.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes AFOG (Attention-Focused Offensive Gradient), a white-box adversarial attack for object detection that iteratively learns an attention map and a perturbation map. The method is evaluated on twelve transformer-based detectors on COCO and on three CNN-based detectors (Faster R-CNN, SSD, YOLOv3) on VOC, reporting benign mAP reductions of up to 37.8x and claiming up to 83% improvement over existing methods, with better speed and imperceptibility. Two variants, AFOG-V (vanishing) and AFOG-F (fabrication), are also introduced. The central technical idea is a learnable, per-pixel attention mask that amplifies or dampens the adversarial perturbation during iterative projected gradient descent.

Significance. If the claims are supported, AFOG would be a useful contribution: it provides a unified attention-guided attack that works on both transformer-based and CNN-based detectors, needs only 10 iterations, and is validated on a broad set of twelve transformer architectures. The paper also makes a favorable reproducibility effort by linking to source code, reporting timing and imperceptibility metrics, and including an ablation showing that the attention mechanism improves attack performance by up to 30.6%. However, the headline 'outperforms existing attacks by up to 83%' is not currently established, because the supporting comparisons use numbers imported from other papers under different threat models and budgets, and because one of the CNN comparisons (Table 4) contains a counterexample where TOG beats AFOG. The self-contained result—that AFOG collapses mAP on all twelve transformers—is credible, but the comparative claim needs either new matched experiments or substantial qualification.

major comments (5)
  1. [§4.2, Table 2] The comparison against GARSDC, GALD, RAD, GHFD, UEA, DAG, RAP, EBAD, AttentionFool, OATB, and DBA is not like-for-like. The table's note 'Results are theirs' and the varied perturbation budgets (e.g., 0.063 vs AFOG's 0.031), iteration counts, and threat model types (surrogate/black-box transfer vs AFOG's white-box victim access) mean that the reported '82.7% improvement' on Swin and the abstract's 'up to 83%' claim are not established. Beating a black-box transfer attack with a white-box attack is expected and does not demonstrate method superiority. To support the claim, AFOG and the baselines should be re-run under identical budgets, iterations, and access settings (or at least a matched white-box subset), with variance reported.
  2. [§4.3, Table 4] Table 4 contains a direct counterexample to the claim that AFOG outperforms existing attacks on CNN-based detectors: on YOLOv3, TOG achieves mAP 0.56 while AFOG achieves 2.28 (the text says 2.62), and the text itself admits 'TOG is a stronger attack' on YOLOv3. Additionally, the FRCNN row is internally inconsistent: Table 4 reports AFOG mAP 2.38, while the text reports 2.07. These discrepancies must be resolved, and the 'outperforms existing attacks' claim should be qualified to the cases where AFOG actually wins (SSD-300 and FRCNN), or the counterexample should be discussed as a limitation.
  3. [§3.2, Equations (6)–(7) and Algorithm 1] There is a sign inconsistency between the update equations and the pseudocode. Equations (6) and (7) write A_{k+1} ← A_k + α_A σ[∂L_AFOG/∂A_k] and P_{k+1} ← P_k + α_P Γ[∂L_AFOG/∂P_k], which is gradient ascent on L_AFOG. Algorithm 1 (lines 14–15) instead writes A_{k+1} ← A_k − α_A L_A and P_{k+1} ← P_k − α_P L_P, which is gradient descent. Since L_AFOG in Equation (3) is defined as a sum of (benign output − adversarial output) terms, gradient descent would decrease the attack loss, not maximize it. This is a load-bearing reproducibility issue; please unify the sign convention and state clearly whether the attack performs ascent or descent.
  4. [§3.4, Equations (8)–(9) and Algorithm 2] For AFOG-V, Algorithm 2 initializes O_x to the empty set ∅. With the definitions in Equations (4) and (5), the sums over i=1..N_x are then empty, so L_bbox = L_cls = 0 and L_AFOG-V = 0, yielding a zero gradient. As written, the AFOG-V procedure cannot produce the reported results in Table 1. The vanishing-objective loss needs to be defined over the actual set of predicted boxes (or otherwise reformulated) so that suppressing all detections corresponds to a nonzero loss. Without this, the AFOG-V experiments are not reproducible from the manuscript.
  5. [§4.1, Table 1] All mAP numbers in Table 1 are point estimates with no variance, standard deviation, or confidence intervals. Since adversarial attacks can be sensitive to the random initialization of P_0 (Equation 2), the 'up to 37.8x reduction' claim would be more convincing if the experiments were repeated over several seeds or if at least a variance estimate were provided. This does not invalidate the main result, but it is needed to assess the stability of the reported reductions.
minor comments (6)
  1. [Supplementary §2.2, Figure 8] The attention learning rate α_A = 0.1 is selected by tuning on three detection transformers. If these are the same victim models later tested in Table 1, this is a form of selection on the test set; please either disclose which models were used for tuning or justify that the chosen α_A generalizes without such tuning.
  2. [Table 3] The µ∆ entry for DETR-R101 under AFOG-V is listed as 0.013, which deviates from the ~0.017 values in all other rows; this is likely a typo and should be corrected.
  3. [§3.1] The formal attack objective is garbled: 'argmax_{x∈D,i∈N_x} , {(b_i, c_i) ∈ R[f_D(ϑ, x_adv)] (IOU(B_i, b_i) < γ ∨ C_i ≠ c_i), min||x − x_adv||_p}' mixes maximization, set membership, and a minimization constraint in a single expression. Please rewrite this as a clearly structured constraint optimization problem.
  4. [Equations (4)–(5)] The notation f_ϑ(x, o_i) is not defined explicitly. It could mean an objectness score, a classification logit, or a bounding-box IoU score, and the subscript 'o_i' versus 'o_adv_i' is ambiguous. Please give precise definitions of these quantities so that the loss is fully specified.
  5. [Abstract] The abstract and the full-text abstract differ on the code link: one says 'Code is available at: Link' and the other gives a URL. Ensure the camera-ready version includes the working URL consistently.
  6. [§4.4] When reporting the attention ablation (Figure 5), please state clearly whether the 30.6% improvement is computed as a relative reduction in mAP (e.g., (mAP_without − mAP_with)/mAP_without) or as an absolute difference, and whether the values refer to AFOG or AFOG-V/AFOG-F.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: AFOG's core attack derivation is self-contained; only minor same-group baseline citations appear.

full rationale

AFOG's core derivation is a standard optimization loop: the attack loss (Eq. 3) is defined directly on the victim detector's bounding-box and class outputs, and the gradients with respect to A and P (Eqs. 6-7) update a multiplicative attention map and a perturbation under an L-infinity projection. The reported mAP collapse across twelve transformers is an independent empirical measurement, not an identity: the loss is a proxy for misdirection, while the mAP is computed by the victim detectors themselves. The only citation-sensitive parts are the imported baseline numbers in Table 2 ('Results are theirs') and the CNN-baseline numbers in Table 4 taken from [8], which is prior work by the same group, plus the attention learning-rate tuning on three victims reported in the supplement. These are fairness and overfitting concerns, not circular reductions: no equation in the paper reduces to its own input by construction, and the self-citations serve as comparison baselines or figure sources rather than load-bearing premises for the attack's design or for the core mAP-collapse result. The derivation is therefore substantively self-contained, with a score of 2 reflecting only minor self-citation rather than actual circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central method rests on standard white-box gradient optimization and on the choice of several hyperparameters (attention learning rate, epsilon, iterations, perturbation step). No new physical or structural entities are introduced; the adversarial attention map is a learned per-pixel mask, not a new model component.

free parameters (4)
  • Attention learning rate alpha_A = 0.1
    Chosen by maximizing attack performance on three detection transformers (Figure 8), then applied to all twelve; not held out.
  • Perturbation budget epsilon = 0.031 (L-infinity)
    Set to match prior attack comparisons; not tuned per model.
  • Number of attack iterations T = 10
    Paper states 10 iterations is effective for all models; not derived.
  • Perturbation learning rate alpha_P = not specified
    Required for Equation 7 but no numeric value is given in the paper, a hidden free parameter.
assumptions (4)
  • domain assumption White-box gradient access to the victim model
    AFOG backpropagates through the victim detector to update A and P (Section 3.2, Equations 6-7); this limits deployment to scenarios where model parameters are exposed.
  • domain assumption Benign predictions serve as ground truth when labels are unavailable
    Section 3.2: 'the attack assumes these benign predictions to be correct labels'; effective only if the victim detector is reasonably accurate.
  • standard math Differentiability of the victim model and loss
    Gradient-based updates require the full forward and backward pass to be differentiable (Section 3.2).
  • domain assumption mAP on the perturbed test set is the correct measure of attack success
    The paper equates attack effectiveness with mAP reduction (Section 4.1); this ignores other measures such as recall/precision tradeoffs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adversarial Attention Perturbations for Large Object Detection Transformers." pith.science (2026). https://pith.science/paper/XYCS6ZYJ

@misc{pith2026250802987,
  author       = {Pith},
  title        = {Pith review of: Adversarial Attention Perturbations for Large Object Detection Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYCS6ZYJ}},
  note         = {Machine review of arXiv:2508.02987}
}
read the original abstract

Adversarial perturbations are useful tools for exposing vulnerabilities in neural networks. Existing adversarial perturbation methods for object detection are either limited to attacking CNN-based detectors or weak against transformer-based detectors. This paper presents an Attention-Focused Offensive Gradient (AFOG) attack against object detection transformers. By design, AFOG is neural-architecture agnostic and effective for attacking both large transformer-based object detectors and conventional CNN-based detectors with a unified adversarial attention framework. This paper makes three original contributions. First, AFOG utilizes a learnable attention mechanism that focuses perturbations on vulnerable image regions in multi-box detection tasks, increasing performance over non-attention baselines by up to 30.6%. Second, AFOG's attack loss is formulated by integrating two types of feature loss through learnable attention updates with iterative injection of adversarial perturbations. Finally, AFOG is an efficient and stealthy adversarial perturbation method. It probes the weak spots of detection transformers by adding strategically generated and visually imperceptible perturbations which can cause well-trained object detection models to fail. Extensive experiments conducted with twelve large detection transformers on COCO demonstrate the efficacy of AFOG. Our empirical results also show that AFOG outperforms existing attacks on transformer-based and CNN-based object detectors by up to 83% with superior speed and imperceptibility. Code is available at https://github.com/zacharyyahn/AFOG.

Figures

Figures reproduced from arXiv: 2508.02987 by the authors.

Figure 1
Figure 1. Successful AFOG attacks against three object detection [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. AFOG attack framework. During attack iteration the ad [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Example attention map generation on DETR-R101 for k=10 iterations. AFOG attention initially learns important regions of the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Visualization with six examples, comparing benign scenario and AFOG attack results on Detection Transformer (DETR) with [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Improvement (% difference) of AFOG with attention [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: DETR self-attention weights analysis. Each blue frame [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Worst case analysis with DETR-R50 as victim model. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: AFOG improvement over iterative for varying values of [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Example images for Benign, AFOG, AFOG-V, and AFOG-F scenarios with victims R50 (DINO) and InternImage (DINO). [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Comparison of Benign and AFOG-disrupted predictions on five example images with ResNet-50, InternImage, FocalNet, and [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Comparison of Benign and AFOG-disrupted predictions on five images with two similar architectures, DETR-R50 and DETR [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Comparison of Benign and AFOG-disrupted predictions on five images with ViTDet, EVA, and DETA. Although AFOG shows [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Comparison of Benign and AFOG-disrupted predictions on five example images with Deformable-DETR, ConvNeXt, and [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Comparison of three images across Benign, AFOG, TOG, UEA, RAP, and DAG cases. AFOG displays superior class and [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Three example images from the VOC dataset where AFOG outperforms TOG. [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 39 canonical work pages

  1. [1]

    Mart ´ın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Man ´e, Rajat Monga, Sherry Moore, Derek M...

    2015

  2. [2]

    Align-detr: Improving detr with sim- ple iou-aware bce loss, 2023

    Zhi Cai, Songtao Liu, Guodong Wang, Zheng Ge, Xiangyu Zhang, and Di Huang. Align-detr: Improving detr with sim- ple iou-aware bce loss, 2023. 5, 6, 7, 11, 15

    work page 2023

  3. [3]

    Salman Asif

    Zikui Cai, Yaoteng Tan, and M. Salman Asif. Ensemble- based blackbox attacks on dense prediction. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4045–4055, 2023. 2, 6

    work page 2023

  4. [4]

    End-to-end object detection with transformers

    Nicolas Carrion, Francisco Massa, Gabriel Synnaeve, Nico- las Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference on Computer Vision, pages 213–223, 2020. 1, 5, 6, 7, 11, 15

    work page 2020

  5. [6]

    Rele- vance attack on detectors

    Sizhe Chen, Fan He, Xiaolin Huang, and Kun Zhang. Rele- vance attack on detectors. Pattern Recognition, 124:108491,

    work page

  6. [7]

    Understanding ob- ject detection through an adversarial lens

    Ka-Ho Chow, Ling Liu, Mehmet Emre Gursoy, Stacey Truex, Wenqi Wei, and Yanzhao Wu. Understanding ob- ject detection through an adversarial lens. In European Sym- posium on Research in Computer Security , pages 460–481. Springer, 2020. 19

    work page 2020

  7. [8]

    Adversarial objectness gradient attacks in real- time object detection systems

    Ka-Ho Chow, Ling Liu, Margaret Loper, Juhyun Bae, Mehmet Emre Gursoy, Stacey Truex, Wenqi Wei, and Yanzhao Wu. Adversarial objectness gradient attacks in real- time object detection systems. In IEEE International Con- ference on Trust, Privacy and Security in Intelligent Systems, and Applications, pages 263–272. IEEE, 2020. 2, 6, 7, 12, 15

    work page 2020

  8. [9]

    Object detection on coco test- dev

    Papers With Code. Object detection on coco test- dev. https://paperswithcode.com/sota/object-detection-on- coco. 1, 12

    work page

Show all 44 references
  1. [10]

    Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, and Andrew Zisserman. The Pascal Visual Object Classes (VOC) Challenge. International Jour- nal of Computer Vision, 88(2):303–338, June 2010. 2, 5

    2010

  2. [11]

    Eva: Exploring the limits of masked visual represen- tation learning at scale

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual represen- tation learning at scale. arXiv preprint arXiv:2211.07636 ,

    arXiv

  3. [12]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Shu. Deep residual learning for image recognition. In Computer Vision and Pattern Recognition, pages 770–778, 06 2016. 5, 6, 7, 11, 15

    2016

  4. [13]

    Object-aware transfer-based black-box adversarial attack on object detector

    Zhuo Leng, Zesen Cheng, Pengxu Wei, and Jie Chen. Object-aware transfer-based black-box adversarial attack on object detector. In Qingshan Liu, Hanzi Wang, Zhanyu Ma, Weishi Zheng, Hongbin Zha, Xilin Chen, Liang Wang, and Rongrong Ji, editors, Pattern Recognition and Computer ...

    2024

  5. [14]

    Improving transferable adversarial attack for vision transformers via global attention and local drop

    Tuo Li and Yahong Han. Improving transferable adversarial attack for vision transformers via global attention and local drop. Multimedia Systems, 29:3467 – 3480, 2023. 2, 6

    2023

  6. [15]

    Exploring plain vision transformer backbones for object de- tection, 2022

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection, 2022. 2, 6, 7, 11, 15

    2022

  7. [16]

    Robust adversarial perturbation on deep proposal-based models

    Yuezun Li, Daniel Tian, Mingching Chang, Xiao Bian, and Siwei Lyu. Robust adversarial perturbation on deep proposal-based models. In BMVC, 2018. 6, 7, 12, 15

    2018

  8. [17]

    Attack anything: Blind dnns via universal background adversarial attack, 2024

    Jiawei Lian, Shaohui Mei, Xiaofei Wang, Yi Wang, Lefan Wang, Yingjie Lu, Mingyang Ma, and Lap-Pui Chau. Attack anything: Blind dnns via universal background adversarial attack, 2024. 2, 4, 6

    2024

  9. [18]

    A large-scale multiple-objective method for black-box attack against object detection

    Siyuan Liang, Longkang Li, Yanbo Fan, and Xiaojun Jia. A large-scale multiple-objective method for black-box attack against object detection. European Conference on Computer Vision (ECCV) 2022, pages 619–636, 2022. 2, 6

    2022

  10. [19]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuyte- laars, editors, ECCV 2014, pages 740–755. Spri...

    2014

  11. [20]

    Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg. Ssd: Single shot multibox detector. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016 , pages 21–37, Cham, 2016. Springe...

    2016

  12. [21]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 1, 2, 5, 6, 7, 12, 15

    2021

  13. [22]

    A convnet for the 2020s, 2022

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s, 2022. 6, 7, 11, 15

    2022

  14. [23]

    Give me your attention: Dot-product attention considered harmful for adversarial patch robustness

    Giulio Lovisotto, Nicole Finnie, Mauricio Munoz, Chaithanya Kumar Murnmadi, and Jan Hendrik Met- zen. Give me your attention: Dot-product attention considered harmful for adversarial patch robustness. In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)...

    2022

  15. [24]

    Towards deep learn- ing models resistant to adversarial attacks

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learn- ing models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. 3

    2018

  16. [25]

    A Survey and Evaluation of Adversarial Attacks for Object Detection, Aug

    Khoi Nguyen Tiet Nguyen, Wenyu Zhang, Kangkang Lu, Yuhuan Wu, Xingjian Zheng, Hui Li Tan, and Liangli Zhen. A Survey and Evaluation of Adversarial Attacks for Object Detection, Aug. 2024. arXiv:2408.01934 [cs]. 2, 6

    2024 arXiv

  17. [26]

    Nms strikes back

    Jeffrey Ouyang-Zhang, Jang Hyun Cho, Xingyi Zhou, and Philipp Kr ¨ahenb¨uhl. Nms strikes back. arXiv preprint arXiv:2212.06137, 2022. 6, 7, 12, 15

    2022 arXiv

  18. [27]

    Pytorch: An im- perative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Rai- son, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, L...

    2019

  19. [28]

    Yolov3: An incremental improvement, 2018

    Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement, 2018. 1, 2, 5, 6, 12

    2018

  20. [29]

    Faster r-cnn: Towards real-time object detection with region proposal networks, 2016

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks, 2016. 1, 6, 12

    2016

  21. [30]

    detrex: Benchmarking de- tection transformers, 2023

    Tianhe Ren, Shilong Liu, Feng Li, Hao Zhang, Ailing Zeng, Jie Yang, Xingyu Liao, Ding Jia, Hongyang Li, He Cao, Jianan Wang, Zhaoyang Zeng, Xianbiao Qi, Yuhui Yuan, Jianwei Yang, and Lei Zhang. detrex: Benchmarking de- tection transformers, 2023. 5, 11

    2023

  22. [31]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Il- lia Polosukhin. Attention is all you need. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett, editors,Advances in N...

    2017

  23. [32]

    Internimage: Exploring large-scale vi- sion foundation models with deformable convolutions.arXiv preprint arXiv:2211.05778, 2022

    Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, et al. Internimage: Exploring large-scale vi- sion foundation models with deformable convolutions.arXiv preprint arXiv:2211.05778, 2022. 6, 7, 12, 15

    2022 arXiv

  24. [33]

    Gradient-guided hierarchical feature attack for object detector

    Yang Wang, Yunfei Zheng, Lei Chen, Zhen Yang, Jingwei Wu, and Tieyong Cao. Gradient-guided hierarchical feature attack for object detector. Journal of King Saud University - Computer and Information Sciences, 36(1):101901, 2024. 2, 6

    2024

  25. [34]

    Transferable adversarial attacks for image and video object detection

    Xingxing Wei, Siyuan Liang, Ning Chen, and Xiaochun Cao. Transferable adversarial attacks for image and video object detection. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 , pages 954–960. International Joint Conferen...

    2019

  26. [35]

    Detectron2

    Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detectron2. https://github. com/facebookresearch/detectron2, 2019. 5, 11

    2019

  27. [36]

    Adversarial examples for se- mantic segmentation and object detection

    Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie1, and Alan Yuill. Adversarial examples for se- mantic segmentation and object detection. In International Conference on Computer Vision, pages 1378–1387, 2017. 6, 7, 12, 15

    2017

  28. [37]

    Focal modulation networks, 2022

    Jianwei Yang, Chunyuan Li, Xiyang Dai, and Jianfeng Gao. Focal modulation networks, 2022. 6, 7, 12, 15

    2022

  29. [38]

    Ni, and Heung-Yeung Shum

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M. Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection, 2022. 5, 11

    2022

  30. [39]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 5, 6, 7, 11, 15 Adversarial Attention Perturbations for Large Object Detection Transformer...

    2010 arXiv

  31. [40]

    • For AFOG’s implementation a link to an anonymous downloadable source repository is included in our ab- stract

    Reproducibility Statement We make the following efforts to enhance the reproducibil- ity of our results. • For AFOG’s implementation a link to an anonymous downloadable source repository is included in our ab- stract. The source includes links for all datasets and models used ...

  32. [41]

    Additional Experimental Setup 2.1. Model Details In our experiments to validate the effectiveness of AFOG at- tacking vision transformer models for object detection, we select twelve transformers of varying model sizes, ranging from Detection Transformer (DETR) [4], a lightwei...

    2017

  33. [43]

    A larger version of InternImage is also cur- rently one of the strongest models on the COCO ob- ject detection leaderboard [9]

    We choose InternImage-Large for our experi- ments to explore AFOG’s applicability to deformable con- volution. A larger version of InternImage is also cur- rently one of the strongest models on the COCO ob- ject detection leaderboard [9]. We use InternImage-Large, a deeper mod...

    2017

  34. [44]

    The key difference between AFOG and AFOG-V is the replacement of Ox with a set of zero predictions ∅ instead of forward propagating image Ox ← fD(x; ϑ)

    AFOG Variants and Pseudocode In addition to the pseudocode for AFOG provided in our main paper, in this section, we also provide the pseudocode for AFOG-V in Algorithm 2 and AFOG-F in Algorithm 3. The key difference between AFOG and AFOG-V is the replacement of Ox with a set o...

  35. [45]

    Couch” prediction in both models by disrupting both class and bounding box losses. Similarly, AFOG induces several small “Cat

    Adversarial Attacks: Visualization In this section, we provide additional visualization of AFOG, AFOG-V , and AFOG-F attacks. Algorithm 2 AFOG-V attack Require: Victim image x ∈ D, test-set D, Victim pre- trained model fD(ϑ), Perturbation step size αP , Atten- tion step size α...

  36. [2017]

    We choose ViTDet for our experiments to investigate AFOG’s potential applicability to all ViT-based models

    Like DINO and DETR, ViT is a widely pop- ular model that has been adapted for numerous uses. We choose ViTDet for our experiments to investigate AFOG’s potential applicability to all ViT-based models. Original code adapted for our implementation is avail- able at https://githu...

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.