REVIEW 3 major objections 4 minor 46 references
Towards Adaptive Meta-Gradient Adversarial Examples for Visual Tracking
T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A black-box meta-gradient attack transfers across visual trackers, cutting SiamRPN++ success from 0.639 to 0.420.
desk verdict Solid attack paper with broad evaluation, but its baseline comparison is under-matched and the attack objective is under-specified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the meta-learning task split over a model repository. In each iteration AMGA randomly selects n+1 ImageNet classifiers; the first n form a meta-training set whose weighted ensemble probability is attacked with cross-entropy loss and momentum-based sign-gradient updates, and the last forms a meta-testing set on which the resulting perturbation receives one further update. Because the ensemble weights are learned and the input is randomly scaled and padded, the perturbation is pushed onto features shared by many surrogate models rather than any single classifier. Gaussian smoothing then removes high-frequency components, which the paper argues improves both transferability and imperceptibility.
What would settle it
Repeat the OTB2015 experiment with Eq. (2)'s label y replaced by a random label or a constant; if success-rate drops stay near AMGA's, the classification objective is not load-bearing, and if they collapse, the transfer claim depends entirely on that surrogate label.
Extended reading notes
Core claim
AMGA is claimed to make black-box adversarial attacks on visual trackers practical by treating a repository of pretrained image classifiers as surrogate models and attacking them through a meta-learning loop. In each learning task, n models are used for meta-training, where a weighted ensemble loss is maximized with sign-gradient updates, and one held-out model is used for meta-testing, where the perturbation is refined once more. The paper reports that this yields adversarial examples that transfer to unseen trackers: on OTB2015 the success rate of SiamRPN++ drops from 0.639 to 0.420 and precision from 0.849 to 0.562, with comparable degradations for SiamCAR, DiMP, MixFormer, TransT, OSTrack, and SeqTrack across three benchmark datasets. The method outperforms the SPARK and IoU black-box baselines while remaining below the white-box CSA baseline.
Load-bearing premise
The load-bearing premise is that maximizing a classification model's loss on the first frame, with a label y that the paper never defines for tracking videos, yields perturbations that transfer to trackers solving a different task.
Editorial extensions
If this is right
- A practical black-box attack on visual trackers needs no tracker gradients: only the first-frame image and a set of public classification models are required.
- A single-frame perturbation can disrupt tracking over entire sequences, since the reported success and precision drops are measured over full benchmark runs.
- The attack transfers across architectural families, covering Siamese trackers (SiamCAR, SiamRPN++), online discriminative trackers (DiMP), and Transformer trackers (MixFormer, TransT, OSTrack, SeqTrack).
- The ablation study attributes the gain to the combination of meta-gradient learning, input diversity, momentum, and Gaussian smoothing, with the full method outperforming each component on its own.
- Among the black-box baselines tested, AMGA outperforms SPARK and IoU on SiamRPN++ / OTB2015, suggesting that transfer-based black-box attacks deserve a place in tracker threat models.
Reading between the lines
- Editorial inference: the label y in Eq. (2) is never defined for tracking frames; if it is an ImageNet category, the method implicitly assumes tracker-relevant feature geometry aligns with classification categories, and randomizing y would test whether that premise is doing the work.
- Editorial inference: at 1.1 fps AMGA is slower than the IoU and SPARK baselines, so its practical use is likely limited to offline precomputation or settings where attack strength matters more than speed.
- Editorial inference: the meta-train/meta-test loop over surrogate models is a generic recipe; the same structure, with momentum and smoothing, could be carried to adversarial attacks on re-identification, dense prediction, or video object segmentation.
- Editorial inference: if the reported transfer depends on ImageNet-pretrained backbones, trackers trained from scratch or on non-ImageNet features may be far less vulnerable; this is testable but not tested in the paper.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes AMGA, a black-box adversarial attack for visual trackers. AMGA generates perturbations on the first frame of a video by optimizing cross-entropy loss of an ensemble of ImageNet classification surrogate models, using a meta-training/meta-testing procedure with random model selection, momentum, Gaussian smoothing, and input diversity. The authors evaluate the attack on seven trackers (SiamCAR, SiamRPN++, DiMP, MixFormer, TransT, OSTrack, SeqTrack) across OTB2015, LaSOT, and GOT-10k, reporting large success-rate and precision drops relative to random noise. They also compare AMGA with CSA, SPARK, and IoU attacks on SiamRPN++/OTB2015, claim superior black-box performance, and provide ablations and visualizations.
Significance. If the empirical claims hold, AMGA would be a useful black-box attack for robustness evaluation of visual trackers, since it demonstrates that perturbations computed on generic ImageNet classifiers can transfer to a wide range of modern trackers. The paper has several strengths: it evaluates on seven trackers and three datasets with consistent large drops versus random noise; the ablation study in Table 4 attributes improvements to the proposed components; and the authors promise code and data at a public repository. However, the central comparative claim against existing black-box attacks is not yet supported because the comparison protocol is not matched and the attack objective is under-specified. The headline results against random noise are plausible, but the claimed superiority over SPARK and IoU requires a controlled experimental setup.
major comments (3)
- [Section 3.2, Eq. (2) and Eq. (8)] The attack objective uses a true label y and dataset D for the cross-entropy loss, but y is never defined for tracking benchmarks. The surrogate models are ImageNet classifiers (Section 4.1), while the trackers predict bounding boxes; no mapping from OTB2015/LaSOT/GOT-10k ground truth to classification labels is given. Please specify how y is constructed (e.g., the target object's ImageNet/WordNet class, or a false class) and justify why maximizing this classification loss on the first frame is an appropriate surrogate for disrupting the tracker's regression and similarity objectives. Without this, the core attack objective is not reproducible.
- [Table 6 and Section 4.1] The black-box comparison with SPARK and IoU is not matched in threat model. The text states that the AMGA perturbation is used on the initial frame only, whereas SPARK is an online incremental attack and IoU is a temporally coherent attack; yet no perturbation budget (epsilon), PSNR, or number of attacked frames is reported for any method. The reported margins over IoU are small (0.219 vs 0.196 success drop; 0.287 vs 0.261 precision drop). A modest change in perturbation magnitude or in the number of attacked frames could reverse the ranking, so the claim that AMGA 'significantly outperforms' these baselines is unsupported as presented.
- [Section 4.1 and Eqs. (3)-(9)] No perturbation budget (epsilon) or projection/clipping step is reported or described anywhere in the methodology or experiments. The update rules in Eqs. (3), (5), and (9) do not constrain the perturbation magnitude, so the attacks in Tables 1-6 are not reproducible and their comparison with standard attacks that use explicit budgets is ambiguous. Please report the l_infinity (or other) constraint used, include any clipping to the epsilon ball, and state the perturbation strength for every configuration, including the Table 6 baselines.
minor comments (4)
- [Table 2] The tracker name 'TansT' in the Table 2 header is a typo and should be 'TransT'.
- [Section 4.3, Table 4] The text says 'Independently applying the input diversity strategy further improved the attack performance,' but the row labeled 'w/ input diversity' may be interpreted as input diversity combined with meta-gradient learning rather than input diversity alone. Please clarify the exact configuration of each ablation row.
- [Figure 2] Several mathematical expressions in Figure 2 appear garbled (subscripts and superscripts are unclear). Please replace them with clean typeset equations that match Eqs. (3)-(5) in the text.
- [Section 4.4] SPARK is described as a 'transfer-based black-box attack,' but the cited paper describes it as a spatial-aware online incremental attack; please verify this characterization and describe the actual attack mechanism accurately.
Circularity Check
No significant circularity; AMGA is optimized on external surrogate classifiers and evaluated on held-out trackers, with only a minor test-set sigma-selection caveat in the black-box comparison.
full rationale
The derivation chain is self-contained and not circular in its core. The attack objective (Eqs. 2-3 and 8-9) is defined over ImageNet classification models in a repository, and the perturbation is optimized without access to the evaluated trackers' parameters or losses. The seven trackers tested in Tables 1-3 are held out from that optimization, so the reported attack drops are external transfer results, not fitted predictions. The paper's self-citations (refs. 4, 6, 8) appear only in background descriptions of tracking methods and are not load-bearing for the AMGA mechanism. No uniqueness theorem, ansatz-smuggling citation, or renaming of a known result is present. One caveat is that Gaussian smoothing sigma is selected on the same SiamRPN++/OTB2015 setup later used in the Table 6 comparison ("Based on the experimental results, we chose a Gaussian smoothing standard deviation of 1 for AMGA"), and Table 6 reports exactly the sigma=1 row from Table 5 (0.219/0.287). This is a test-set hyperparameter-selection concern that weakens the headline black-box comparison, but it is not a construction-level equivalence: the sigma choice does not define the surrogate objective or the tracker evaluations, and the transferability claims across seven trackers and three datasets remain externally measured rather than derived from the same fit. The undefined label y for tracking frames is an objective-validity gap, not a circularity. Overall, no step reduces the claimed results to their own inputs by definition or by fitted-parameter renaming.
Assumptions & free parameters
free parameters (5)
- alpha (update step size) =
0.01
- mu (momentum coefficient) =
0.9
- sigma (Gaussian smoothing standard deviation) =
1.0
- beta_i (ensemble weights) =
learned, values not reported
- perturbation budget (epsilon) =
not specified
assumptions (4)
- domain assumption Cross-entropy gradients from ImageNet classification models transfer to visual tracker failures.
- domain assumption Attacking only the initial frame is sufficient to degrade whole-video tracking metrics.
- domain assumption Randomly selecting n+1 models and taking one extra gradient step constitutes meta-learning that improves transferability.
- standard math Standard sign-based iterative gradient updates and Gaussian convolution are valid operations.
Cite this review
Pith. "Pith review of Towards Adaptive Meta-Gradient Adversarial Examples for Visual Tracking." pith.science (2026). https://pith.science/paper/6TN35VIB
@misc{pith2026250508999,
author = {Pith},
title = {Pith review of: Towards Adaptive Meta-Gradient Adversarial Examples for Visual Tracking},
year = {2026},
howpublished = {\url{https://pith.science/paper/6TN35VIB}},
note = {Machine review of arXiv:2505.08999}
}
read the original abstract
In recent years, visual tracking methods based on convolutional neural networks and Transformers have achieved remarkable performance and have been successfully applied in fields such as autonomous driving. However, the numerous security issues exposed by deep learning models have gradually affected the reliable application of visual tracking methods in real-world scenarios. Therefore, how to reveal the security vulnerabilities of existing visual trackers through effective adversarial attacks has become a critical problem that needs to be addressed. To this end, we propose an adaptive meta-gradient adversarial attack (AMGA) method for visual tracking. This method integrates multi-model ensembles and meta-learning strategies, combining momentum mechanisms and Gaussian smoothing, which can significantly enhance the transferability and attack effectiveness of adversarial examples. AMGA randomly selects models from a large model repository, constructs diverse tracking scenarios, and iteratively performs both white- and black-box adversarial attacks in each scenario, optimizing the gradient directions of each model. This paradigm minimizes the gap between white- and black-box adversarial attacks, thus achieving excellent attack performance in black-box scenarios. Extensive experimental results on large-scale datasets such as OTB2015, LaSOT, and GOT-10k demonstrate that AMGA significantly improves the attack performance, transferability, and deception of adversarial examples. Codes and data are available at https://github.com/pgao-lab/AMGA.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
F. Chen, X. Wang, Y . Zhao, S. Lv, X. Niu, Visual object tracking: A survey, Computer Vision and Image Understanding 222 (2022) 103508
work page 2022
-
[2]
V . Jadeja, A. Rao, A. Srivastava, S. Singh, P. Chaturvedi, G. Bhardwaj, Convolutional neural networks: A comprehensive review of architectures and application, in: 2023 6th International Conference on Contemporary Computing and Informatics (IC3I), V ol. 6, IEEE, 2023, pp. 460–467
work page 2023
- [3]
-
[4]
P. Gao, Q. Zhang, F. Wang, L. Xiao, H. Fujita, Y . Zhang, Learning reinforced attentional representation for end-to-end visual tracking, Information Sciences 517 (2020) 52–67
work page 2020
-
[5]
L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, P. H. Torr, Fully-convolutional siamese networks for object tracking, in: Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II 14, Springer, 2016, pp. 850–865
work page 2016
-
[6]
P. Gao, R. Yuan, F. Wang, L. Xiao, H. Fujita, Y . Zhang, Siamese attentional keypoint network for high performance visual tracking, Knowledge-based systems 193 (2020) 105448
work page 2020
-
[7]
B. Li, J. Yan, W. Wu, Z. Zhu, X. Hu, High performance visual tracking with siamese region proposal network, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2018, pp. 8971–8980. 16
work page 2018
- [8]
Show all 46 references
-
[9]
I. J. Goodfellow, J. Shlens, C. Szegedy, Explaining and harnessing adversarial examples, arXiv preprint arXiv:1412.6572
-
[10]
Kurakin, I
A. Kurakin, I. J. Goodfellow, S. Bengio, Adversarial examples in the physical world, in: Artificial intelligence safety and security, Chapman and Hall/CRC, 2018, pp. 99–112
2018
-
[11]
Zhang, Y
L. Zhang, Y . Zhou, Y . Yang, X. Gao, Meta invariance defense towards generalizable robustness to unknown adversarial attacks, IEEE Transactions on Pattern Analysis and Machine Intelligence
-
[12]
Hospedales, A
T. Hospedales, A. Antoniou, P. Micaelli, A. Storkey, Meta-learning in neural networks: A survey, IEEE transactions on pattern analysis and machine intelligence 44 (9) (2021) 5149–5169
2021
-
[13]
Szegedy, V
C. Szegedy, V . Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2016, pp. 2818–2826
2016
-
[14]
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2016, pp. 770–778
2016
-
[15]
Sandler, A
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L.-C. Chen, Mobilenetv2: Inverted residuals and linear bottlenecks, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2018, pp. 4510–4520
2018
-
[16]
Huang, Z
G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger, Densely connected convolutional net- works, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2017, pp. 4700–4708
2017
-
[17]
Simonyan, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556
K. Simonyan, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556
-
[18]
Krizhevsky, I
A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, Communications of the ACM 60 (6) (2017) 84–90
2017
-
[19]
F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, K. Keutzer, Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <0.5mb model size, arXiv preprint arXiv:1602.07360
-
[20]
Zagoruyko, Wide residual networks, arXiv preprint arXiv:1605.07146
S. Zagoruyko, Wide residual networks, arXiv preprint arXiv:1605.07146
-
[21]
D. Guo, J. Wang, Y . Cui, Z. Wang, S. Chen, Siamcar: Siamese fully convolutional classification and regression for visual tracking, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 6269–6277
2020
-
[22]
B. Li, W. Wu, Q. Wang, F. Zhang, J. Xing, J. Yan, Siamrpn++: Evolution of siamese visual tracking with very deep networks, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4282–4291
2019
-
[23]
G. Bhat, M. Danelljan, L. V . Gool, R. Timofte, Learning discriminative model prediction for tracking, in: Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 6182–6191
2019
-
[24]
Y . Cui, C. Jiang, L. Wang, G. Wu, Mixformer: End-to-end tracking with iterative mixed attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 13608–13618
2022
-
[25]
X. Chen, B. Yan, J. Zhu, D. Wang, X. Yang, H. Lu, Transformer tracking, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 8126–8135
2021
-
[26]
B. Ye, H. Chang, B. Ma, S. Shan, X. Chen, Joint feature learning and relation modeling for tracking: A one-stream framework, in: European conference on computer vision, Springer, 2022, pp. 341–357
2022
-
[27]
X. Chen, H. Peng, D. Wang, H. Lu, H. Hu, Seqtrack: Sequence to sequence learning for visual object tracking, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 14572–14581
2023
-
[28]
Y . Wu, J. Lim, M.-H. Yang, Object tracking benchmark, IEEE Transactions on Pattern Analysis and Machine Intelligence 37 (09) (2015) 1834–1848. 17
2015
-
[29]
Huang, X
L. Huang, X. Zhao, K. Huang, Got-10k: A large high-diversity benchmark for generic object tracking in the wild, IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (5) (2019) 1562–1577
2019
-
[30]
H. Fan, H. Bai, L. Lin, F. Yang, P. Chu, G. Deng, S. Yu, M. Huang, J. Liu, Y . Xu, et al., Lasot: A high-quality large-scale single object tracking benchmark, International Journal of Computer Vision 129 (2) (2021) 439–461
2021
-
[31]
Vaswani, Attention is all you need, Advances in Neural Information Processing Systems
A. Vaswani, Attention is all you need, Advances in Neural Information Processing Systems
-
[32]
Zhang, A
L. Zhang, A. Gonzalez-Garcia, J. V . D. Weijer, M. Danelljan, F. S. Khan, Learning the model update for siamese trackers, in: Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 4010–4019
2019
-
[33]
Danelljan, L
M. Danelljan, L. V . Gool, R. Timofte, Probabilistic regression for visual tracking, in: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 7183–7192
2020
-
[34]
Tramer, A
F. Tramer, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, P. McDaniel, Ensemble adversarial training: Attacks and defenses, arXiv preprint arXiv:1705.07204
-
[35]
Y . Dong, F. Liao, T. Pang, X. Hu, J. Zhu, Discovering adversarial examples with momentum, arXiv preprint arXiv:1710.06081 5
-
[36]
K. Ren, T. Zheng, Z. Qin, X. Liu, Adversarial attacks and defenses in deep learning, Engineering 6 (3) (2020) 346–360
2020
-
[37]
M. Lei, H. Song, J. Fan, D. Xiao, D. Ai, Y . Gu, J. Yang, Gaa: Ghost adversarial attack for object tracking, IEEE Transactions on Emerging Topics in Computational Intelligence
-
[38]
S. Jia, C. Ma, Y . Song, X. Yang, M.-H. Yang, Robust deep object tracking against adversarial attacks, International Journal of Computer Vision (2024) 1–20
2024
-
[39]
Liang, X
S. Liang, X. Wei, S. Yao, X. Cao, Efficient adversarial attacks for visual object tracking, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVI 16, Springer, 2020, pp. 34–50
2020
-
[40]
S. Zhao, T. Xu, X.-J. Wu, J. Kittler, Pluggable attack for visual object tracking, IEEE Transac- tions on Information Forensics and Security 19 (2023) 1227–1240
2023
-
[41]
X. Chen, X. Yan, F. Zheng, Y . Jiang, S.-T. Xia, Y . Zhao, R. Ji, One-shot adversarial attacks on visual tracking with dual attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10176–10185
2020
-
[42]
Q. Guo, Z. Cheng, F. Juefei-Xu, L. Ma, X. Xie, Y . Liu, J. Zhao, Learning to adversarially blur visual object tracking, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10839–10848
2021
-
[43]
Z. Yuan, J. Zhang, Y . Jia, C. Tan, T. Xue, S. Shan, Meta gradient adversarial attack, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 7748– 7757
2021
-
[44]
B. Yan, D. Wang, H. Lu, X. Yang, Cooling-shrinking attack: Blinding the tracker with imper- ceptible noises, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 990–999
2020
-
[45]
Q. Guo, X. Xie, F. Juefei-Xu, L. Ma, Z. Li, W. Xue, W. Feng, Y . Liu, Spark: Spatial-aware online incremental attack against visual tracking, in: European conference on computer vision, Springer, 2020, pp. 202–219
2020
-
[46]
S. Jia, Y . Song, C. Ma, X. Yang, Iou attack: Towards temporally coherent black-box adversarial attack for visual object tracking, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 6709–6718. 18
2021
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.