The Scissors Effect: When Resize-Based Input Diversity Helps or Hurts Transfer Attacks
Pith reviewed 2026-06-26 10:34 UTC · model grok-4.3
The pith
Resize-based input diversity improves adversarial transfer from standard surrogates but reduces it from robust ones.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Increasing the DI probability raises transfer success for standard surrogates but lowers it for robust ones; the two response curves separate like a pair of scissors. A controlled robustness-strength sweep that varies only the training budget shows the harm is graded rather than binary. A resize/translation decomposition attributes roughly 67 percent of the harm to resize, and direct source-target gradient-alignment measurements confirm that the same resize operation improves alignment for standard surrogates but degrades it for robust ones. Local Gradient Consistency (LGC) separates the two surrogate types, and a bias-variance crossover theorem isolates where DI helps from where its resize
What carries the argument
The Scissors Effect, produced by the resize component of input diversity altering source-target gradient alignment in opposite directions according to surrogate robustness.
If this is right
- Blind use of DI reduces attack success by 10.3 percent on average when the source is robustly trained.
- The sign of DI's effect on transfer flips from positive to negative as robustness increases from low to moderate levels.
- Roughly two-thirds of the performance loss on robust surrogates is attributable to the resize operation rather than padding.
- A training-free rule that disables diversity when LGC is high preserves DI's benefit on standard surrogates while avoiding its cost on robust ones.
Where Pith is reading between the lines
- Attack pipelines may need to treat input diversity as conditional on surrogate properties rather than always-on.
- The same gradient-alignment mechanism that produces the Scissors Effect could be checked for other common attack ingredients such as momentum or ensemble methods.
- Defenses that increase model robustness might simultaneously reduce vulnerability to attacks that rely on diversity.
Load-bearing premise
The controlled robustness-strength sweep that varies only the training budget isolates robustness as the sole variable driving the sign flip in DI response.
What would settle it
Measuring transfer success rates while sweeping DI probability on matched pairs of standard and robust surrogates, with all other attack settings fixed, would directly test whether the performance curves separate as described.
Figures
read the original abstract
Input Diversity (DI), which applies random resizing and padding at each attack iteration, is a near-default ingredient of transfer-based adversarial attacks, widely assumed to improve transferability. We show this assumption is regime-dependent and, for robustly trained surrogates, often reversed. Varying only the surrogate, increasing the DI probability raises transfer success for standard surrogates but lowers it for robust ones: the two response curves separate like a pair of scissors, a pattern we call the Scissors Effect. The effect is strong and consistent on ImageNet, where blind DI costs the robust source 10.3% attack success on average across CNN, ViT, Swin, and ConvNeXt targets and across ten attacks spanning 2018-2024; it is smaller on CIFAR-10 unless DI is made aggressive. A controlled robustness-strength sweep that varies only the training budget shows the harm is graded rather than binary, crossing from beneficial to harmful already in the little-robustness regime. We trace it to gradient geometry: a resize/translation decomposition attributes roughly 67% of the harm to resize, and a direct source-target gradient-alignment measurement confirms the same resize operation improves alignment for standard surrogates but degrades it for robust ones. We summarize the regime with Local Gradient Consistency (LGC), a single input-space probe that separates the two surrogate types, and prove a bias-variance crossover theorem isolating where DI helps from where its resize bias dominates. A training-free rule (CG-DI) that disables diversity when LGC is high avoids the loss on robust surrogates while keeping DI's benefit on standard ones, positioning the Scissors Effect as a DI-specific manifestation of the broader robustness-transferability trade-off.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that resize-and-pad input diversity (DI), a default component of transfer attacks, exhibits a 'Scissors Effect': increasing DI probability improves transfer success for standard surrogates but reduces it for robustly trained ones. On ImageNet this produces an average 10.3% drop in attack success for robust sources across CNN/ViT/Swin/ConvNeXt targets and ten attacks; the effect is smaller on CIFAR-10 unless DI is aggressive. The sign flip is shown to be graded via a training-budget sweep, traced to gradient geometry (resize/translation decomposition attributes ~67% of harm to resize; direct alignment measurements confirm the pattern), summarized by a Local Gradient Consistency (LGC) probe, and isolated by a bias-variance crossover theorem. A training-free CG-DI rule that disables diversity when LGC is high is proposed to retain DI benefits on standard surrogates while avoiding harm on robust ones.
Significance. If the empirical separation and the isolation of robustness as the driving variable hold, the result would materially affect transfer-attack design by showing that a near-universal ingredient is regime-dependent and by supplying a simple LGC-based correction. The consistent pattern across four architectures and ten attacks (2018-2024), the direct gradient-alignment tests, and the training-free CG-DI rule constitute concrete, actionable strengths. The work also supplies a concrete instance of the broader robustness-transferability trade-off localized to the resize operation.
major comments (3)
- [Abstract / §4] Abstract and §4 (robustness sweep): the claim that 'varying only the training budget' isolates robustness as the sole variable driving the sign flip in DI response is load-bearing for the graded-harm and scissors-effect conclusions, yet no convergence diagnostics (training/validation loss curves, gradient-norm statistics, or feature-learning metrics) are reported to confirm that optimization state and loss-landscape curvature remain comparable across budgets. Without these checks the attribution risks confounding with convergence quality.
- [Abstract] Abstract: the bias-variance crossover theorem is invoked to 'isolate where DI helps from where its resize bias dominates,' but neither the theorem statement, its assumptions, nor its proof appear in the provided text; this prevents verification that the theorem actually separates the two regimes independently of the empirical measurements.
- [Results] Results (ImageNet 10.3% figure): the reported average cost of blind DI on robust sources is presented without per-attack or per-architecture standard errors, confidence intervals, or statistical tests, making it impossible to assess whether the 'strong and consistent' claim is robust to sampling variation across the ten attacks.
minor comments (2)
- The definition and exact computation of Local Gradient Consistency (LGC) are introduced without an explicit equation or pseudocode in the main text, complicating reproduction of the CG-DI rule.
- Figure captions and axis labels for the DI-probability sweeps should explicitly state the number of random seeds and whether error bars represent standard deviation or standard error.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed comments. We address each major point below and commit to revisions that directly strengthen the empirical and theoretical claims.
read point-by-point responses
-
Referee: [Abstract / §4] Abstract and §4 (robustness sweep): the claim that 'varying only the training budget' isolates robustness as the sole variable driving the sign flip in DI response is load-bearing for the graded-harm and scissors-effect conclusions, yet no convergence diagnostics (training/validation loss curves, gradient-norm statistics, or feature-learning metrics) are reported to confirm that optimization state and loss-landscape curvature remain comparable across budgets. Without these checks the attribution risks confounding with convergence quality.
Authors: We agree that convergence diagnostics are necessary to rule out confounding by optimization state. In the revised manuscript we will add training and validation loss curves together with gradient-norm statistics across the training-budget sweep, confirming that models at different budgets reach comparable convergence levels before the DI experiments are run. revision: yes
-
Referee: [Abstract] Abstract: the bias-variance crossover theorem is invoked to 'isolate where DI helps from where its resize bias dominates,' but neither the theorem statement, its assumptions, nor its proof appear in the provided text; this prevents verification that the theorem actually separates the two regimes independently of the empirical measurements.
Authors: The bias-variance crossover theorem is central to the theoretical contribution. Its full statement, assumptions, and proof were omitted from the submitted version. We will insert the complete theorem (including assumptions and a self-contained proof) into the revised manuscript, placed in the main text or an appendix so that the separation of regimes can be verified independently of the experiments. revision: yes
-
Referee: [Results] Results (ImageNet 10.3% figure): the reported average cost of blind DI on robust sources is presented without per-attack or per-architecture standard errors, confidence intervals, or statistical tests, making it impossible to assess whether the 'strong and consistent' claim is robust to sampling variation across the ten attacks.
Authors: We acknowledge that variability measures are required to substantiate the consistency claim. The revised version will report per-attack and per-architecture standard errors (or confidence intervals) for the 10.3 % average and will include appropriate statistical tests comparing DI versus no-DI conditions across the attack suite. revision: yes
Circularity Check
No circularity: empirical sweeps and theorem are self-contained
full rationale
The paper's claims rest on controlled empirical measurements (DI probability sweeps across surrogate types, gradient-alignment tests, LGC probe) and a stated bias-variance crossover theorem. No step reduces by the paper's own equations to a fitted parameter defined from the target result, nor relies on load-bearing self-citation or ansatz smuggled via prior work. The robustness sweep is presented as isolating training budget; even if confounding factors exist, they do not constitute circularity under the specified criteria. The derivation chain is independent of the reported outcomes.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption ImageNet and CIFAR-10 with the listed model families and attack methods from 2018-2024 are representative for measuring transfer success.
invented entities (2)
-
Local Gradient Consistency (LGC)
no independent evidence
-
CG-DI rule
no independent evidence
Reference graph
Works this paper leans on
-
[1]
International Conference on Machine Learning (ICML) , pages =
Synthesizing robust adversarial examples , author =. International Conference on Machine Learning (ICML) , pages =
-
[2]
IEEE Symposium on Security and Privacy (SP) , pages =
Towards evaluating the robustness of neural networks , author =. IEEE Symposium on Security and Privacy (SP) , pages =
-
[3]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
Unlabeled data improves adversarial robustness , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =
-
[4]
Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track , year =
RobustBench: A standardized adversarial robustness benchmark , author =. Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track , year =
-
[5]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Boosting adversarial attacks with momentum , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[6]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Evading defenses to transferable adversarial examples by translation-invariant attacks , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[7]
International Conference on Learning Representations (ICLR) , year =
An image is worth 16x16 words: Transformers for image recognition at scale , author =. International Conference on Learning Representations (ICLR) , year =
-
[8]
2019 , url =
Robustness (Python Library) , author =. 2019 , url =
2019
-
[9]
Proceedings of the 20th ACM Asia Conference on Computer and Communications Security (ASIACCS) , pages =
Transferable adversarial examples with Bayesian approach , author =. Proceedings of the 20th ACM Asia Conference on Computer and Communications Security (ASIACCS) , pages =
-
[10]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
Boosting adversarial transferability by achieving flat local maxima , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =
-
[11]
International Conference on Learning Representations (ICLR) , year =
Explaining and harnessing adversarial examples , author =. International Conference on Learning Representations (ICLR) , year =
-
[12]
arXiv preprint arXiv:2010.03593 , year =
Uncovering the limits of adversarial training against norm-bounded adversarial examples , author =. arXiv preprint arXiv:2010.03593 , year =
-
[13]
International Conference on Learning Representations (ICLR) , year =
Countering adversarial images using input transformations , author =. International Conference on Learning Representations (ICLR) , year =
-
[14]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Deep residual learning for image recognition , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[15]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Densely connected convolutional networks , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[16]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
Adversarial examples are not bugs, they are features , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =
-
[17]
International Conference on Learning Representations (ICLR) , year =
Nesterov accelerated gradient and scale invariance for adversarial attacks , author =. International Conference on Learning Representations (ICLR) , year =
-
[18]
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =
Swin Transformer: Hierarchical vision transformer using shifted windows , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =
-
[19]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
A ConvNet for the 2020s , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[20]
Proceedings of the AAAI Conference on Artificial Intelligence , volume =
On the convergence of an adaptive momentum method for adversarial attacks , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =
-
[21]
European Conference on Computer Vision (ECCV) , pages =
Frequency domain model augmentation for adversarial attack , author =. European Conference on Computer Vision (ECCV) , pages =
-
[22]
International Conference on Learning Representations (ICLR) , year =
Towards deep learning models resistant to adversarial attacks , author =. International Conference on Learning Representations (ICLR) , year =
-
[23]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
When adversarial training meets vision transformers: Recipes from training to architecture , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =
-
[24]
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Transferability in machine learning: from phenomena to black-box attacks using adversarial samples , author =. arXiv preprint arXiv:1605.07277 , year =
work page internal anchor Pith review Pith/arXiv arXiv
-
[25]
Robust principles: Architectural design principles for adversarially robust
Peng, ShengYun and Xu, Weilin and Cornelius, Cory and Hull, Matthew and Li, Kevin and Duggal, Rahul and Phute, Mansi and Martin, Jason and Chau, Duen Horng , booktitle =. Robust principles: Architectural design principles for adversarially robust
-
[26]
International Conference on Machine Learning (ICML) , pages =
Learning transferable visual models from natural language supervision , author =. International Conference on Machine Learning (ICML) , pages =
-
[27]
International Conference on Machine Learning (ICML) , pages =
Overfitting in adversarially robust deep learning , author =. International Conference on Machine Learning (ICML) , pages =
-
[28]
Do adversarially robust
Salman, Hadi and Ilyas, Andrew and Engstrom, Logan and Kapoor, Ashish and Madry, Aleksander , booktitle =. Do adversarially robust
-
[29]
International Conference on Learning Representations (ICLR) , year =
Robust learning meets generative models: Can proxy distributions improve adversarial robustness? , author =. International Conference on Learning Representations (ICLR) , year =
-
[30]
International Conference on Learning Representations (ICLR) , year =
Very deep convolutional networks for large-scale image recognition , author =. International Conference on Learning Representations (ICLR) , year =
-
[31]
International Conference on Learning Representations (ICLR) , year =
Intriguing properties of neural networks , author =. International Conference on Learning Representations (ICLR) , year =
-
[32]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Rethinking the Inception architecture for computer vision , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[33]
International Conference on Learning Representations (ICLR) , year =
Ensemble adversarial training: Attacks and defenses , author =. International Conference on Learning Representations (ICLR) , year =
-
[34]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
High-frequency component helps explain the generalization of convolutional neural networks , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[35]
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =
Admix: Enhancing the transferability of adversarial attacks , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =
-
[36]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Enhancing the transferability of adversarial attacks through variance tuning , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[37]
Proceedings of the 40th International Conference on Machine Learning (ICML) , pages =
Better diffusion models further improve adversarial training , author =. Proceedings of the 40th International Conference on Machine Learning (ICML) , pages =
-
[38]
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =
Structure invariant transformation for better adversarial transferability , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =
-
[39]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Boosting adversarial transferability by block shuffle and rotation , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[40]
IEEE Transactions on Information Forensics and Security , year =
Improving transferable targeted adversarial attack via normalized logit calibration and truncated feature mixing , author =. IEEE Transactions on Information Forensics and Security , year =
-
[41]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Improving transferability of adversarial examples with input diversity , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[42]
Communications in Computational Physics , volume =
Frequency principle: Fourier analysis sheds light on deep neural networks , author =. Communications in Computational Physics , volume =
-
[43]
IEEE Transactions on Information Forensics and Security , volume =
Quantization aware attack: Enhancing transferable adversarial attacks by model quantization , author =. IEEE Transactions on Information Forensics and Security , volume =
-
[44]
International Conference on Machine Learning (ICML) , pages =
Theoretically principled trade-off between robustness and accuracy , author =. International Conference on Machine Learning (ICML) , pages =
-
[45]
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =
Boosting adversarial transferability via gradient relevance attack , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =
-
[46]
Torchattacks: A
Kim, Hoki , journal =. Torchattacks: A
-
[47]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
A little robustness goes a long way: Leveraging robust features for targeted transfer attacks , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =
-
[48]
IEEE Symposium on Security and Privacy (SP) , pages =
Why does little robustness help? A further step towards understanding adversarial transferability , author =. IEEE Symposium on Security and Privacy (SP) , pages =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.