pith. sign in

arxiv: 2605.20308 · v1 · pith:CNWH2J3Cnew · submitted 2026-05-19 · 💻 cs.CV · cs.AI· cs.LG

SDM: A Powerful Tool for Evaluating Model Robustness

Xinlei Liu , Tao Hu , Jichao Xie , Peng Yi , Hailong Ma , Baolin Li This is my paper

Pith reviewed 2026-05-21 07:46 UTC · model grok-4.3

classification 💻 cs.CV cs.AIcs.LG
keywords adversarial attacksmodel robustness evaluationgradient-based methodsSequential Difference Maximizationcomputer visionoptimization frameworks
0
0 comments X

The pith

SDM improves adversarial attack performance and efficiency by reconstructing the objective to maximize probability differences between labels.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that previous gradient-based attacks often produce high-loss examples that are not adversarial because of unsuitable objectives for generating adversarial examples. By analyzing this issue, the authors reconstruct the objective as maximizing the difference between the upper bound of non-ground-truth label probabilities and the ground-truth label probability. They then propose Sequential Difference Maximization (SDM), which uses a three-layer optimization framework to sequentially optimize this objective with tailored loss functions in different stages. This results in attacks that are stronger and more cost-effective than prior state-of-the-art methods. A sympathetic reader would care because better attacks allow for more accurate evaluation of how robust machine learning models are to adversarial perturbations.

Core claim

The paper establishes that the high-loss non-adversarial examples issue stems from inappropriate objectives, and that reformulating the objective to maximize the difference between the non-ground-truth label probability upper bound and the ground-truth label probability, pursued via the Sequential Difference Maximization method in a cycle-stage-step framework with negative probability loss and Directional Probability Difference Ratio loss, produces adversarial examples with stronger attack performance and superior cost-effectiveness compared to previous methods.

What carries the argument

The Sequential Difference Maximization (SDM) method, which implements a three-layer 'cycle-stage-step' optimization framework using negative probability loss in the initial stage and Directional Probability Difference Ratio (DPDR) loss in subsequent stages to approach the ideal adversarial objective.

If this is right

  • SDM achieves stronger attack performance than previous state-of-the-art methods such as APGD.
  • SDM exhibits superior cost-effectiveness in generating adversarial examples.
  • The reconstructed objective directly addresses the high-loss non-adversarial examples problem.
  • Models can be evaluated for robustness more reliably using these improved attacks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Adopting SDM as a standard evaluation tool could lead to the discovery of previously undetected vulnerabilities in deployed computer vision models.
  • Similar sequential optimization strategies might be applied to other adversarial or optimization problems in machine learning to improve efficiency.
  • Further experiments on diverse datasets beyond those tested could validate the generalizability of the performance gains.

Load-bearing premise

That the high-loss non-adversarial examples problem is the main cause of poor performance in prior methods and that the new reconstructed objective solves it without introducing significant new drawbacks.

What would settle it

Running SDM and prior methods like APGD on the same set of models and datasets, measuring attack success rate and computational cost, and finding that SDM does not outperform would falsify the superiority claims.

Figures

Figures reproduced from arXiv: 2605.20308 by Baolin Li, Hailong Ma, Jichao Xie, Peng Yi, Tao Hu, Xinlei Liu.

Figure 1
Figure 1. Figure 1: Probability landscapes of standard adversarial examples and high-loss non-adversarial examples. 4, x ′ (1) and x ′ (2) are two adversarial examples generated under the perturbation budget of ℓ∞ = 8/255. When x ′ (1) and x ′ (2) are fed into the classification model, their output performances are presented in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overall structure of SDM. sub-objectives. It performs sequential optimization on these sub-objectives (i.e., the optimal solution of the previous stage is used as the initial solution for the subsequent stage), thereby gradually approaching the optimal solution of the overall optimization objective. 4.2.1 Overall Structure As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Intersection and difference proportions of adversarial examples generated by various attack methods. the results demonstrate that they represent distinct techni￾cal pathways, with no significant generational gap or clear performance advantage observed between them. In [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: , APGD exhibits the steepest performance growth curve, with its effectiveness stabilizing only after 500 itera￾tions. Consequently, when computational cost is severely constrained (e.g., total iterations ≤ 20), its performance will be significantly inadequate. In contrast, SDM achieves supe￾rior performance across all iteration counts, demonstrating a clear advantage. This indicates that SDM offers the hig… view at source ↗
Figure 6
Figure 6. Figure 6: Attack success rates of various attack methods under different interferences with a ℓ∞-norm constraint. are launched against the defense method VAT using PGD, C&W, APGD and the proposed SDM. Next, the four afore￾mentioned interference effects are separately applied to the generated raw adversarial examples. Both the raw adver￾sarial examples and the interference-added adversarial ex￾amples are input into t… view at source ↗
Figure 7
Figure 7. Figure 7: Attack success rates of various attack methods under different interferences with a ℓ2-norm constraint. which in turn impairs their adversarial performance. By con￾trast, the proposed SDM continues to achieve the highest anti-interference capability under the ℓ2-norm constraint. In summary, the adversarial examples generated by SDM under the constraints of the two involved norms (ℓ∞ and ℓ2) possess relativ… view at source ↗
read the original abstract

Gradient-based attacks are important methods for evaluating model robustness. However, since the proposal of APGD, it has been difficult for such methods to achieve significant breakthroughs. To achieve such an effect, we first analyze the issue of "high-loss non-adversarial examples" that degrades attack performance in previous methods, and prove that this issue arises from inappropriate objectives for adversarial example generation. Subsequently, we reconstruct the objective as "maximizing the difference between the non-ground-truth label probability upper bound and the ground-truth label probability", and proposes a novel and powerful gradient-based attack method named Sequential Difference Maximization (SDM). SDM establishes a three-layer optimization framework of "cycle-stage-step". It adopts the negative probability loss function and the Directional Probability Difference Ratio (DPDR) loss function in the initial and subsequent optimization stages, respectively, and approaches the ideal objective of adversarial example generation via stage-wise sequential optimization. Experiments demonstrate that compared with previous state-of-the-art methods, SDM not only achieves stronger attack performance but also exhibits superior cost-effectiveness. The code is available at https://github.com/X-L-Liu/ICML-SDM.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper claims that gradient-based adversarial attacks have stagnated since APGD due to the 'high-loss non-adversarial examples' problem, which it attributes to and proves arises from inappropriate objectives. It reconstructs the objective as maximizing the difference between the non-ground-truth label probability upper bound and the ground-truth label probability. The proposed Sequential Difference Maximization (SDM) uses a three-layer 'cycle-stage-step' optimization framework, applying negative probability loss in the initial stage and the Directional Probability Difference Ratio (DPDR) loss in subsequent stages to approach the ideal objective. Experiments show SDM achieves stronger attack performance and superior cost-effectiveness versus prior SOTA methods.

Significance. If the central empirical claims hold after addressing attribution issues, this would be a solid incremental advance in adversarial attack design for robustness evaluation. The explicit objective reconstruction and staged framework offer a structured alternative to prior methods, and the public code release supports reproducibility.

major comments (1)
  1. [Experiments] Experiments section: The paper reports that SDM outperforms previous SOTA in attack strength and cost-effectiveness, but the comparisons omit ablations isolating the three-layer 'cycle-stage-step' framework from the new objective and loss functions. No results are provided for non-staged variants (e.g., single-stage DPDR loss or single-stage negative probability loss) under identical iteration budgets and models. This makes it difficult to attribute the performance gains specifically to the reconstructed objective rather than the added staging machinery, directly affecting support for the central claim of stronger attack performance.
minor comments (2)
  1. [Abstract] The abstract states that a proof is provided for the origin of the high-loss issue but does not reference the specific section containing the proof or derivation, which would aid readers in locating the technical details.
  2. [Methods] The definition and motivation for the Directional Probability Difference Ratio (DPDR) loss function could include an explicit equation or pseudocode in the methods section to clarify its relation to the reconstructed objective.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive feedback on our manuscript. We appreciate the suggestion to strengthen the attribution of performance gains through additional ablations and address the point below.

read point-by-point responses

  1. Referee: [Experiments] Experiments section: The paper reports that SDM outperforms previous SOTA in attack strength and cost-effectiveness, but the comparisons omit ablations isolating the three-layer 'cycle-stage-step' framework from the new objective and loss functions. No results are provided for non-staged variants (e.g., single-stage DPDR loss or single-stage negative probability loss) under identical iteration budgets and models. This makes it difficult to attribute the performance gains specifically to the reconstructed objective rather than the added staging machinery, directly affecting support for the central claim of stronger attack performance.

    Authors: We agree that explicit ablations would help isolate the contribution of the staged optimization from the reconstructed objective and loss functions. The 'cycle-stage-step' framework is central to SDM because it enables sequential refinement: the initial stage uses negative probability loss to escape poor local optima, while subsequent stages apply DPDR loss to approach the ideal objective of maximizing the probability difference. A purely single-stage approach would not implement this progressive strategy. To directly address the concern, we will add results for single-stage DPDR and single-stage negative probability loss variants (using identical iteration budgets, models, and evaluation protocols) in the revised manuscript. These ablations will clarify the role of staging in the observed gains. revision: yes

Circularity Check

0 steps flagged

No circularity: objective reconstruction and staged framework are independent proposals

full rationale

The paper identifies a performance issue in prior gradient-based attacks, attributes it to objective choice via analysis, then explicitly reconstructs a new target objective and introduces a three-layer cycle-stage-step optimizer with negative probability loss followed by DPDR loss. These elements are presented as novel design choices rather than quantities derived from or fitted to prior results within the paper. No self-citation chain, uniqueness theorem, or renaming of known patterns is invoked to justify the central construction. Empirical superiority claims rest on direct comparisons under the new framework, not on any reduction of outputs to inputs by construction. The derivation chain therefore remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 1 invented entities

The central claim depends on standard gradient-descent assumptions plus the validity of the new objective reconstruction and the staged loss schedule; one new loss function is introduced without external validation.

free parameters (1)
  • Stage transition thresholds and step counts
    The three-layer cycle-stage-step framework requires choices for when to switch losses and how many iterations per stage.
axioms (1)
  • domain assumption Gradient information remains informative for maximizing the probability-difference objective across stages
    Invoked when proposing the sequential optimization approach.
invented entities (1)
  • Directional Probability Difference Ratio (DPDR) loss function no independent evidence
    purpose: To refine the attack in later optimization stages toward the ideal objective
    Newly defined component of the method with no independent evidence supplied in the abstract.

pith-pipeline@v0.9.0 · 5742 in / 1276 out tokens · 35682 ms · 2026-05-21T07:46:56.660609+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

  1. [1]

    Adversarial Examples

    Costas Neocleous and Christos N. Schizas , editor =. On the Claim for the Existence of "Adversarial Examples" in Deep Learning Neural Networks , booktitle =. 2014 , url =. doi:10.5220/0005152503060309 , timestamp =

    work page doi:10.5220/0005152503060309 2014

  2. [2]

    Goodfellow and Jonathon Shlens and Christian Szegedy , editor =

    Ian J. Goodfellow and Jonathon Shlens and Christian Szegedy , editor =. Explaining and Harnessing Adversarial Examples , booktitle =. 2015 , url =

    work page 2015

  3. [3]

    Advances in Adversarial Attacks and Defenses in Computer Vision:

    Naveed Akhtar and Ajmal Mian and Navid Kardan and Mubarak Shah , author =. Advances in Adversarial Attacks and Defenses in Computer Vision:. 2021 , url =. doi:10.1109/ACCESS.2021.3127960 , timestamp =

    work page doi:10.1109/access.2021.3127960 2021

  4. [4]

    Jiangfan Liu and Yishan Li and Yanming Guo and Yu Liu and Jun Tang and Ying Nie , title =. Artif. Intell. Rev. , volume =. 2024 , url =. doi:10.1007/S10462-024-10841-Z , timestamp =

    work page doi:10.1007/s10462-024-10841-z 2024

  5. [5]

    Proceedings of the 37th International Conference on Machine Learning,

    Francesco Croce and Matthias Hein , title =. Proceedings of the 37th International Conference on Machine Learning,. 2020 , url =

    work page 2020

  6. [6]

    Wagner , title =

    Nicholas Carlini and David A. Wagner , title =. 2017. 2017 , url =. doi:10.1109/SP.2017.49 , timestamp =

    work page doi:10.1109/sp.2017.49 2017

  7. [7]

    DeepFool:

    Seyed. DeepFool:. 2016. 2016 , url =. doi:10.1109/CVPR.2016.282 , timestamp =

    work page doi:10.1109/cvpr.2016.282 2016

  8. [8]

    6th International Conference on Learning Representations,

    Aleksander Madry and Aleksandar Makelov and Ludwig Schmidt and Dimitris Tsipras and Adrian Vladu , title =. 6th International Conference on Learning Representations,. 2018 , url =

    work page 2018

  9. [9]

    Adversarial Attacks and Countermeasures on Image Classification-based Deep Learning Models in Autonomous Driving Systems:

    Bakary Badjie and Jos. Adversarial Attacks and Countermeasures on Image Classification-based Deep Learning Models in Autonomous Driving Systems:. 2025 , url =. doi:10.1145/3691625 , timestamp =

    work page doi:10.1145/3691625 2025

  10. [10]

    Goodfellow and Rob Fergus , editor =

    Christian Szegedy and Wojciech Zaremba and Ilya Sutskever and Joan Bruna and Dumitru Erhan and Ian J. Goodfellow and Rob Fergus , editor =. Intriguing properties of neural networks , booktitle =. 2014 , url =

    work page 2014

  11. [11]

    Goodfellow and Samy Bengio , title =

    Alexey Kurakin and Ian J. Goodfellow and Samy Bengio , title =. 5th International Conference on Learning Representations,. 2017 , url =

    work page 2017

  12. [12]

    Wide Residual Networks , booktitle =

    Sergey Zagoruyko and Nikos Komodakis , editor =. Wide Residual Networks , booktitle =. 2016 , url =

    work page 2016

  13. [13]

    2009 , month=apr, url=

    Learning Multiple Layers of Features from Tiny Images , author=. 2009 , month=apr, url=

    work page 2009

  14. [14]

    Kaiming He and Xiangyu Zhang and Shaoqing Ren and Jian Sun , title =. 2016. 2016 , url =. doi:10.1109/CVPR.2016.90 , timestamp =

    work page doi:10.1109/cvpr.2016.90 2016

  15. [15]

    8th International Conference on Learning Representations,

    Yisen Wang and Difan Zou and Jinfeng Yi and James Bailey and Xingjun Ma and Quanquan Gu , title =. 8th International Conference on Learning Representations,. 2020 , url =

    work page 2020

  16. [16]

    2024 , url =

    Xiangyu Yin and Wenjie Ruan , title =. 2024 , url =. doi:10.1109/CVPR52733.2024.02317 , timestamp =

    work page doi:10.1109/cvpr52733.2024.02317 2024

  17. [17]

    Reducing Excessive Margin to Achieve a Better Accuracy vs

    Rahul Rade and Seyed. Reducing Excessive Margin to Achieve a Better Accuracy vs. Robustness Trade-off , booktitle =. 2022 , url =

    work page 2022

  18. [18]

    Proceedings of the Thirty-Second

    Pin. Proceedings of the Thirty-Second. 2018 , url =. doi:10.1609/AAAI.V32I1.11302 , timestamp =

    work page doi:10.1609/aaai.v32i1.11302 2018

  19. [19]

    Matching Networks for One Shot Learning , booktitle =

    Oriol Vinyals and Charles Blundell and Tim Lillicrap and Koray Kavukcuoglu and Daan Wierstra , editor =. Matching Networks for One Shot Learning , booktitle =. 2016 , url =

    work page 2016

  20. [20]

    Bartoldson and James Diffenderfer and Konstantinos Parasyris and Bhavya Kailkhura , title =

    Brian R. Bartoldson and James Diffenderfer and Konstantinos Parasyris and Bhavya Kailkhura , title =. Forty-first International Conference on Machine Learning,. 2024 , url =

    work page 2024

  21. [21]

    Square Attack:

    Maksym Andriushchenko and Francesco Croce and Nicolas Flammarion and Matthias Hein , editor =. Square Attack:. Computer Vision -. 2020 , url =. doi:10.1007/978-3-030-58592-1\_29 , timestamp =

    work page doi:10.1007/978-3-030-58592-1 2020

  22. [22]

    Yiyun Zhou and Meng Han and Liyuan Liu and Jing He and Xi Gao , title =. 16th. 2019 , url =. doi:10.1109/MASSW.2019.00012 , timestamp =

    work page doi:10.1109/massw.2019.00012 2019

  23. [23]

    Survey of computer vision algorithms and applications for unmanned aerial vehicles , journal =

    Abdulla Al. Survey of computer vision algorithms and applications for unmanned aerial vehicles , journal =. 2018 , url =. doi:10.1016/J.ESWA.2017.09.033 , timestamp =

    work page doi:10.1016/j.eswa.2017.09.033 2018

  24. [24]

    Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual , year =

    Francesco Croce and Maksym Andriushchenko and Vikash Sehwag and Edoardo Debenedetti and Nicolas Flammarion and Mung Chiang and Prateek Mittal and Matthias Hein , title =. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual , year =

    work page 2021

  25. [25]

    2026 , issn =

    A state transition-based method for influence evaluation in networks , journal =. 2026 , issn =. doi:https://doi.org/10.1016/j.chaos.2025.117713 , url =

    work page doi:10.1016/j.chaos.2025.117713 2026

  26. [26]

    ImageNet:

    Jia Deng and Wei Dong and Richard Socher and Li. ImageNet:. 2009. 2009 , url =. doi:10.1109/CVPR.2009.5206848 , timestamp =

    work page doi:10.1109/cvpr.2009.5206848 2009