Pith. sign in

REVIEW 3 major objections 4 minor 31 references

Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GEAR-SAM reallocates SAM's fixed perturbation budget by historical gradient energy and beats SAM by up to 0.84 on CIFAR-100.

desk verdict Smart, cheap SAM variant with correct math and good CIFAR-100 results; transfer gains are inflated by an unmatched perturbation radius. read the letter →

arxiv 2607.18306 v1 pith:MQI6RGAH submitted 2026-07-17 cs.LG cs.AI

Zhen Huang , Jiaxin Deng , Junbiao Pang This is my paper
classification cs.LGcs.AI
keywords Sharpness-AwareMinimizationblock-wiseperturbationgradientenergyexponentialmovingaveragebudgetallocationflatminimalabelnoiserobustnessHessianspectrum
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that Sharpness-Aware Minimization (SAM) already performs an implicit, block-wise allocation of its perturbation budget, but it does so using only the current minibatch gradient, which is noisy and ignores how each network block's sensitivity changes over training. GEAR-SAM replaces that instantaneous signal with an exponential moving average of squared block-gradient norms, treating this 'gradient energy' as a cheap curvature-related proxy, and derives a closed-form rule that reallocates the same global radius across blocks. The result is a perturbation of no larger norm than SAM's that reaches flatter minima: on CIFAR-100, GEAR-SAM improves accuracy over SAM by 0.66, 0.82, and 0.84 points on ResNet-18, WideResNet-28-10, and PyramidNet-110, and it lowers the largest Hessian eigenvalue found by SAM. The broader claim is that a fixed sharpness-perturbation budget should be dynamically redistributed as functional blocks' roles evolve, not kept at fixed or purely instantaneous shares.

What carries the argument

The load-bearing object is the block-wise gradient-energy score s_{b,t}, the EMA of squared ℓ₂ norms of the gradient restricted to each functional block. From these scores the allocation rule r*_b = ρ s_b / sqrt(Σ_j s_j²) (Proposition 1) distributes the fixed SAM radius ρ. The rule is the exact maximizer of the linear alignment surrogate Σ_b s_b r_b subject to Σ r_b² ≤ ρ², proved by Cauchy–Schwarz; because EMA is linear, the score is equivalent to summing adaptive-gradient-style per-parameter second moments within a block. The architecture-aware partition (stem, coarse residual stages, classifier; normalization affine parameters kept with their block) supplies the blocks and stabilizes the s

What would settle it

Measure, on a training run, the directional block curvatures q_b = u_b^T H_bb u_b at the perturbed directions and correlate them with the EMA scores s_b used by GEAR-SAM; if blocks that receive more budget do not have systematically higher curvature — or if a run with the allocation rule reversed (more budget to low-energy blocks) still matches GEAR-SAM's accuracy — then the proposed mechanism, not the general idea of redistribution, is what needs revision.

Watch

Extended reading notes

Core claim

The paper sets out to establish that the practical SAM perturbation, while optimal for the first-order sharpness objective, ignores curvature-related block sensitivity because it allocates each block a radius proportional to its instantaneous gradient norm. GEAR-SAM instead maintains B scalar scores — an exponential moving average of squared block-gradient norms — and allocates the fixed SAM radius by r_b = ρ s_b / sqrt(Σ_j s_j²), which uniquely maximizes the linear surrogate Σ_b s_b r_b under the budget constraint Σ r_b² ≤ ρ². This preserves the global SAM budget, never enlarges the SAM neighborhood, requires no Hessian-vector products or explicit Fisher estimation, and adds only B scalar s

Load-bearing premise

The load-bearing premise is that an EMA of squared block-gradient norms tracks the block-level curvature that matters for flatness, and that the linear alignment rule in Eq. (20) is a good way to spend the fixed SAM budget; the paper itself labels Eq. (20) a budget-preserving surrogate, not an exact second-order solution.

Editorial extensions

If this is right

  • Because the total perturbation norm is never larger than SAM's, any accuracy gain comes purely from where the budget is spent, not from a stronger sharpness penalty; GEAR-SAM can be applied as a drop-in replacement for SAM with no extra forward-backward passes.
  • The method reduces worst-case local sharpness: on ResNet-18/CIFAR-100 the maximum Hessian eigenvalue drops from 200.72 (SAM) to 192.92, while the top-50 mean stays roughly comparable.
  • The gains concentrate on harder, finer-grained tasks: CIFAR-100 gains are 0.66–0.84 points over SAM while CIFAR-10 differences are near zero, suggesting historical sensitivity matters most when representations must support finer discrimination.
  • Under symmetric label noise, GEAR-SAM outperforms SAM at all four tested noise rates on CIFAR-10 and at 40%, 60%, and 80% on CIFAR-100, with particularly large margins at high noise.
  • The benefit is not tied to one partition: coarse, fine, and tensor-wise groupings all beat SAM on CIFAR-100, with the default coarse blocks nearly matching the best fine-grained split.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable corollary the authors leave implicit: if the mechanism is correct, GEAR-SAM's advantage over SAM should track the heterogeneity of block sensitivities — datasets or architectures where all blocks have similar gradient energy should show near-zero gains, while those with skewed energy should show large gains. That is directly measurable from the EMA scores.
  • The same budget-redistribution principle could be combined with other perturbation schemes, such as direction-changing or layer-wise-scaled variants; nothing in the derivation forces the in-block direction to be the raw gradient, so a natural extension is to use GEAR-SAM's radii with learned or adaptive directions.
  • Because the score is an EMA, the method is a continuous, soft analogue of sparse or selective perturbation methods: instead of deciding whether to perturb a block, it lets blocks fade in and out of the budget. This suggests a calibration test — comparing GEAR-SAM's block share trajectories with the pruning importance of blocks — to see whether the energy signal is tracking representational importa
  • The Hessian spectrum result predicts the benefit shows up under worst-case perturbations, not average-case curvature; a concrete check is adversarial robustness or sharpness under random directions, where GEAR-SAM should show larger margins relative to SAM.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes GEAR-SAM, a modification of Sharpness-Aware Minimization (SAM) that maintains an exponential moving average (EMA) of squared block-gradient norms and uses this signal to allocate a fixed global perturbation radius across parameter blocks through the closed-form solution in Eq. (21). The authors prove that standard SAM implicitly performs a block-wise radius allocation, derive an exact identity for the minibatch gradient second moment (Eq. (15)), and present a budget-preserving allocation rule (Eq. (20)). Experiments cover CIFAR-10/100 image classification, transfer learning on five datasets, label-noise robustness, partition studies, and Hessian spectra, with the stated conclusion that GEAR-SAM improves generalization and robustness across architectures and tasks.

Significance. The paper's derivations are mathematically sound and its main CIFAR-100 results, which use matched perturbation radii, show consistent gains over SAM (0.66, 0.82, and 0.84 points on ResNet-18, WideResNet-28-10, and PyramidNet-110, respectively) with small standard deviations. The method is lightweight, preserves the global SAM budget by construction, and the Hessian top-eigenvalue measurement provides direct evidence of flatter minima. However, the transfer-learning comparison is confounded by unequal perturbation radii, the label-noise table lacks repeated runs, and the central allocation surrogate in Eq. (20) is only loosely connected to the second-order objective it is supposed to approximate. These issues currently prevent the paper from fully supporting its cross-task claims.

major comments (3)
  1. [IV-D (Table III)] The transfer-learning experiments use a perturbation radius of 0.05 for SAM and 0.2 for GEAR-SAM. In squared radius, this is a 16x budget difference (0.04 vs. 0.0025). The large gains on Flowers102 (78.04 to 84.04), Stanford Cars (75.75 to 80.54), and Oxford-IIIT Pet (88.30 to 91.91) could therefore be entirely a global-radius effect rather than an effect of the block-wise allocation mechanism. Since the paper claims GEAR-SAM 'preserves the global SAM budget' and the transfer results are a stated pillar of the 'across architectures and tasks' claim, a matched-budget comparison is required. Please report SAM with rho=0.2 and/or GEAR-SAM with rho=0.05, and ideally a small sweep of rho for both methods.
  2. [IV-C (Table II)] The label-noise robustness table reports single-run accuracies with no standard deviations, despite the image-classification tables reporting three seeds. Moreover, SAM's CIFAR-10 accuracy at 80% noise (76.94) is higher than its accuracy at 60% noise (67.22), which is non-monotonic and unusual for symmetric label noise; this calls for explanation or suggests an unstable evaluation. Since robustness is one of the paper's stated contributions, at least three repeated runs per configuration (or another accepted uncertainty measure) and a discussion of the non-monotonicity are needed before this evidence can support the claim.
  3. [III-D (Eq. (20))] The allocation rule is the solution of a linear surrogate (maximize sum s_b r_b subject to sum r_b^2 <= rho^2), not of the second-order sharpness objective in Eq. (12). The paper explicitly acknowledges this, but the central mechanism -- that EMA squared-gradient energy is a curvature-related sensitivity signal -- is not directly validated. The Hessian spectrum in Fig. 2 is a useful global check, but it does not show that the allocated budget shares track block-wise curvature. Please add an analysis connecting the allocated radii to block-wise curvature (e.g., diagonal Hessian trace or directional curvature per block) or ablations showing that this particular surrogate is preferable to plausible alternatives. Without that, the method's explanation remains a heuristic alongside the empirical results.
minor comments (4)
  1. [III-A and IV-E] The paper says GEAR-SAM introduces 'no additional block-wise hyperparameters' and only scalar state; this is technically true, but the EMA factor beta and the stability constant delta are still additional global hyperparameters selected by tuning. Please state this explicitly to avoid understating the tuning burden.
  2. [IV-A (Table I)] Many baseline numbers are taken directly from the BSAM paper [9]. Please state whether those baselines were run under the same learning rate, weight decay, and epoch schedule as GEAR-SAM; differences in protocols can materially affect the comparisons.
  3. [Appendix / Eq. (15)] The statement that the common factor m cancels after block normalization is correct but brief; a reader may wonder why the factor does not affect the normalized allocation. One clarifying sentence would help.
  4. [General] No code or reproducibility statement is provided. For a method whose contribution is an algorithmic rule, making the implementation available (or at least specifying the exact block partition for each architecture) would significantly strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: GEAR-SAM is a designed surrogate validated empirically; self-citations are not load-bearing.

full rationale

The derivation chain is non-circular. The SAM block-radius decomposition (Eq. 10) and budget preservation are algebraic identities following from the stated objectives and partition; they do not assume the conclusion. The proposed EMA score (Eq. 17) and allocation rule (Eq. 21) are explicit design choices, not derived from a uniqueness theorem or from the experiments. The paper itself disclaims a stronger derivation: Section III-D says the requirements 'do not uniquely solve the second-order inner problem' and calls Eq. (20) 'a budget-preserving surrogate rather than an exact second-order solution,' and the appendix says the KKT expression 'does not derive GEAR-SAM's surrogate allocation rule.' No fitted parameter is renamed as a prediction: rho and beta are tuned and reported as hyperparameters, and the claimed accuracy and Hessian gains are measured outcomes rather than consequences of the construction. The only self-citation is BSAM [9], by two of the present authors, used for baseline numbers and the label-noise protocol; it does not provide any load-bearing assumption for GEAR-SAM's mechanism. The transfer-learning comparison does use different global radii for SAM (0.05) vs. GEAR-SAM (0.2), which is a possible empirical confound, but that is a correctness/experimental-design concern, not circularity: the allocation rule does not reduce to those results. No specific circular step can be exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central method rests on one ad hoc surrogate objective and one domain assumption about gradient-energy as curvature proxy; the paper is transparent about both. No free parameters beyond SAM's ρ, the EMA factor β, and the stability constant δ are introduced, and no new physical/model entities are postulated.

free parameters (3)
  • EMA factor β = 0.9 (default)
    Controls how much history is used in sensitivity scores; parameter study in Table V selects 0.9 on CIFAR-100/ResNet-18, and the same value is used elsewhere without per-task tuning.
  • global SAM radius ρ = 0.1 (CIFAR-10), 0.2 (CIFAR-100), 0.05 (noisy-label), 0.2 transfer GEAR-SAM vs 0.05 transfer SAM
    The global budget is a pre-existing SAM hyperparameter, but GEAR-SAM's gains depend on it; in transfer learning the method uses a 4x larger radius than the SAM baseline, confounding the comparison.
  • stability constant δ = unspecified small positive
    Used to stabilize zero-gradient block directions; the value is not reported, which minorly affects the exact perturbation norm.
assumptions (4)
  • standard math Trust-region optimality theorem (Moré-Sorensen) for quadratic maximization over a norm ball
    Used in Section III-A and the appendix to characterize the ideal second-order perturbation Eq. (6).
  • domain assumption Second-order Taylor expansion of the minibatch loss with cross-block Hessian terms omitted (Eq. 12)
    Used only to motivate why curvature-related block sensitivity should matter; not used in the allocation rule itself.
  • domain assumption EMA of squared block gradients is a usable curvature-related sensitivity signal (empirical-Fisher / expected-Hessian connection)
    Load-bearing for the method; the paper cites [10,11] but also acknowledges empirical Fisher is not generally the Hessian, so this is an unproven design premise.
  • ad hoc to paper Linear alignment surrogate Σ s_b r_b with budget Σ r_b² ≤ ρ² is a good allocation rule (Eq. 20)
    The paper states the requirements 'do not uniquely solve the second-order inner problem' and calls Eq. (20) a budget-preserving surrogate rather than an exact solution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization." pith.science (2026). https://pith.science/paper/MQI6RGAH

@misc{pith2026260718306,
  author       = {Pith},
  title        = {Pith review of: Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MQI6RGAH}},
  note         = {Machine review of arXiv:2607.18306}
}
read the original abstract

Sharpness-Aware Minimization (SAM) improves generalization by minimizing the worst-case loss in a local parameter neighborhood. Standard SAM implicitly allocates its global perturbation budget across parameter blocks according to instantaneous minibatch gradient norms. Such an allocation can be noisy and may not reflect the sensitivity that blocks accumulate throughout training. We propose Gradient-Energy Adaptive Radius SAM (GEAR-SAM), which maintains an exponential moving average (EMA) of squared block gradients as a lightweight, curvature-related sensitivity signal and allocates the fixed SAM budget through a closed-form constrained optimization. GEAR-SAM preserves the global SAM radius, requires no Hessian-vector products or explicit Fisher estimation, and adds only scalar state beyond SAM. Experiments on image classification, transfer learning, noisy-label learning, and partition studies demonstrate improved generalization and robustness across architectures and tasks. More broadly, GEAR-SAM provides a dynamic view of sharpness-aware optimization: a fixed perturbation budget should be redistributed as the sensitivity of functional network blocks evolves during training.

Figures

Figures reproduced from arXiv: 2607.18306 by the authors.

Figure 1
Figure 1. Early block-wise perturbation dynamics of SAM and GEAR-SAM on ResNet-18 trained on CIFAR-100. The plots show the first 40 epochs for the [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Histograms of the top-50 Hessian eigenvalues of ResNet-18 trained on CIFAR-100 with (a) SGD, (b) SAM, and (c) GEAR-SAM. Each panel also [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Full block-wise perturbation dynamics of SAM and GEAR-SAM on ResNet-18 trained on CIFAR-100. Each panel shows the normalized perturbation [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 11 linked inside Pith

  1. [1]

    On large-batch training for deep learning: Generalization gap and sharp minima,

    N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, “On large-batch training for deep learning: Generalization gap and sharp minima,”arXiv preprint arXiv:1609.04836, 2016

    arXiv 2016

  2. [2]

    Fantastic generalization measures and where to find them,

    Y . Jiang, B. Neyshabur, H. Mobahi, D. Krishnan, and S. Bengio, “Fantastic generalization measures and where to find them,”arXiv preprint arXiv:1912.02178, 2019

    arXiv 1912

  3. [3]

    Entropy-sgd: Biasing gradient descent into wide valleys,

    P. Chaudhari, A. Choromanska, S. Soatto, Y . LeCun, C. Baldassi, C. Borgs, J. Chayes, L. Sagun, and R. Zecchina, “Entropy-sgd: Biasing gradient descent into wide valleys,”Journal of Statistical Mechanics: Theory and Experiment, vol. 2019, no. 12, p. 124018, 2019

    2019

  4. [4]

    Sharpness-aware minimization for efficiently improving generalization,

    P. Foret, A. Kleiner, H. Mobahi, and B. Neyshabur, “Sharpness-aware minimization for efficiently improving generalization,”arXiv preprint arXiv:2010.01412, 2020

    arXiv 2010

  5. [5]

    Asam: Adaptive sharpness- aware minimization for scale-invariant learning of deep neural net- works,

    J. Kwon, J. Kim, H. Park, and I. K. Choi, “Asam: Adaptive sharpness- aware minimization for scale-invariant learning of deep neural net- works,” inInternational conference on machine learning. PMLR, 2021, pp. 5905–5914

    2021

  6. [6]

    Surrogate gap minimization improves sharpness-aware training,

    J. Zhuang, B. Gong, L. Yuan, Y . Cui, H. Adam, N. Dvornek, S. Tatikonda, J. Duncan, and T. Liu, “Surrogate gap minimization improves sharpness-aware training,”arXiv preprint arXiv:2203.08065, 2022

    arXiv 2022

  7. [7]

    Fisher sam: Information geometry and sharpness aware minimisation,

    M. Kim, D. Li, S. X. Hu, and T. Hospedales, “Fisher sam: Information geometry and sharpness aware minimisation,” inInternational Confer- ence on Machine Learning. PMLR, 2022, pp. 11 148–11 161. 8

    2022

  8. [8]

    Friendly sharpness- aware minimization,

    T. Li, P. Zhou, Z. He, X. Cheng, and X. Huang, “Friendly sharpness- aware minimization,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 5631–5640

    2024

Show all 31 references
  1. [9]

    Bilateral sharpness- aware minimization for flatter minima,

    J. Deng, J. Pang, B. Zhang, and Q. Huang, “Bilateral sharpness- aware minimization for flatter minima,”IEEE Transactions on Neural Networks and Learning Systems, 2026

    2026

  2. [10]

    Limitations of the empirical fisher approximation for natural gradient descent,

    F. Kunstner, P. Hennig, and L. Balles, “Limitations of the empirical fisher approximation for natural gradient descent,”Advances in neural information processing systems, vol. 32, 2019

    2019

  3. [11]

    Fadam: Adam is a natural gradient optimizer using diagonal empirical fisher information,

    D. Hwang, “Fadam: Adam is a natural gradient optimizer using diagonal empirical fisher information,”arXiv preprint arXiv:2405.12807, 2024

    2024 arXiv

  4. [12]

    Efficient sharpness-aware minimization for improved training of neural networks,

    J. Du, H. Yan, J. Feng, J. T. Zhou, L. Zhen, R. S. M. Goh, and V . Y . Tan, “Efficient sharpness-aware minimization for improved training of neural networks,”arXiv preprint arXiv:2110.03141, 2021

    2021 arXiv

  5. [13]

    Towards efficient and scalable sharpness-aware minimization,

    Y . Liu, S. Mai, X. Chen, C.-J. Hsieh, and Y . You, “Towards efficient and scalable sharpness-aware minimization,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12 360–12 370

    2022

  6. [14]

    An adaptive policy to em- ploy sharpness-aware minimization,

    W. Jiang, H. Yang, Y . Zhang, and J. Kwok, “An adaptive policy to em- ploy sharpness-aware minimization,”arXiv preprint arXiv:2304.14647, 2023

    2023 arXiv

  7. [15]

    K-sam: Sharpness-aware minimization at the speed of sgd,

    R. Ni, P.-y. Chiang, J. Geiping, M. Goldblum, A. G. Wilson, and T. Goldstein, “K-sam: Sharpness-aware minimization at the speed of sgd,”arXiv preprint arXiv:2210.12864, 2022

    2022 arXiv

  8. [16]

    Sampa: Sharpness-aware minimiza- tion parallelized,

    W. Xie, T. Pethick, and V . Cevher, “Sampa: Sharpness-aware minimiza- tion parallelized,”Advances in Neural Information Processing Systems, vol. 37, pp. 51 333–51 357, 2024

    2024

  9. [17]

    Normalization layers are all that sharpness-aware minimization needs,

    M. Mueller, T. Vlaar, D. Rolnick, and M. Hein, “Normalization layers are all that sharpness-aware minimization needs,”Advances in Neural Information Processing Systems, vol. 36, pp. 69 228–69 252, 2023

    2023

  10. [18]

    The crucial role of normalization in sharpness-aware minimization,

    Y . Dai, K. Ahn, and S. Sra, “The crucial role of normalization in sharpness-aware minimization,”Advances in Neural Information Pro- cessing Systems, vol. 36, pp. 67 741–67 770, 2023

    2023

  11. [19]

    µP 2: Effective sharpness aware minimization requires layerwise perturbation scaling,

    M. Haas, J. Xu, V . Cevher, and L. C. Vankadara, “µP 2: Effective sharpness aware minimization requires layerwise perturbation scaling,” arXiv preprint arXiv:2411.00075, 2024

    2024 arXiv

  12. [20]

    Make sharpness-aware minimization stronger: A sparsified perturbation ap- proach,

    P. Mi, L. Shen, T. Ren, Y . Zhou, X. Sun, R. Ji, and D. Tao, “Make sharpness-aware minimization stronger: A sparsified perturbation ap- proach,”Advances in Neural Information Processing Systems, vol. 35, pp. 30 950–30 962, 2022

    2022

  13. [21]

    Improving sharpness-aware minimization with fisher mask for better generalization on language models,

    Q. Zhong, L. Ding, L. Shen, P. Mi, J. Liu, B. Du, and D. Tao, “Improving sharpness-aware minimization with fisher mask for better generalization on language models,” inFindings of the Association for Computational Linguistics: EMNLP 2022, 2022, pp. 4064–4085

    2022

  14. [22]

    Layer-wise adaptive gradient norm penalizing method for efficient and accurate deep learning,

    S. Lee, “Layer-wise adaptive gradient norm penalizing method for efficient and accurate deep learning,” inProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 1518–1529

    2024

  15. [23]

    Sparse layer sharpness-aware minimization for efficient fine- tuning,

    Y . Cheng, X. Yang, G. Wang, C. Huang, F. Ma, D. Yu, X. Cao, and L. Shen, “Sparse layer sharpness-aware minimization for efficient fine- tuning,”arXiv preprint arXiv:2602.09395, 2026

    2026

  16. [24]

    Three factors influencing minima in sgd,

    S. Jastrz˛ ebski, Z. Kenton, D. Arpit, N. Ballas, A. Fischer, Y . Bengio, and A. Storkey, “Three factors influencing minima in sgd,”arXiv preprint arXiv:1711.04623, 2017

    2017 arXiv

  17. [25]

    Sharpness-aware minimization and the edge of stability,

    P. M. Long and P. L. Bartlett, “Sharpness-aware minimization and the edge of stability,”Journal of Machine Learning Research, vol. 25, no. 179, pp. 1–20, 2024

    2024

  18. [26]

    Gradient norm aware minimization seeks first-order flatness and improves generalization,

    X. Zhang, R. Xu, H. Yu, H. Zou, and P. Cui, “Gradient norm aware minimization seeks first-order flatness and improves generalization,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 20 247–20 257

    2023

  19. [27]

    Cr-sam: Curvature regularized sharpness-aware minimization,

    T. Wu, T. Luo, and D. C. Wunsch II, “Cr-sam: Curvature regularized sharpness-aware minimization,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 2024, pp. 6144–6152

    2024

  20. [28]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  21. [29]

    Computing a trust region step,

    J. J. Moré and D. C. Sorensen, “Computing a trust region step,”SIAM Journal on Scientific and Statistical Computing, vol. 4, no. 3, pp. 553– 572, 1983

    1983

  22. [30]

    Pyhessian: Neural networks through the lens of the hessian,

    Z. Yao, A. Gholami, K. Keutzer, and M. W. Mahoney, “Pyhessian: Neural networks through the lens of the hessian,” in2020 IEEE international conference on big data (Big data). IEEE, 2020, pp. 581– 590

    2020

  23. [31]

    O2u-net: A simple noisy label detection approach for deep neural networks,

    J. Huang, L. Qu, R. Jia, and B. Zhao, “O2u-net: A simple noisy label detection approach for deep neural networks,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3326–3334. 9 APPENDIX This appendix provides additional details supporting th...

    2019

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.