{"id":"62669078-b09d-4d9b-85ef-6e7197bc1509","arxiv_id":"2607.18306","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"GEAR-SAM re-allocates SAM's fixed perturbation radius across network blocks in proportion to an EMA of squared block-gradient norms, improving generalization on CIFAR, transfer, and label-noise benchmarks.","lead":"This paper proposes GEAR-SAM, a tweak to Sharpness-Aware Minimization that splits SAM's fixed perturbation budget across network blocks based on a smoothed average of gradient magnitudes. On image classification, transfer learning, and noisy-label benchmarks it reports consistent accuracy gains over SAM while adding almost no computational cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transfer-learning gains are confounded: GEAR-SAM is run at ρ=0.2 while SAM is run at ρ=0.05, so Table III may show a global-radius effect, not allocation.","rationale":"The reader's weakest_assumption concerns the theoretical status of the linear surrogate in Eq. (20): whether EMA gradient energy tracks curvature and whether maximizing Σ s_b r_b is the right allocation objective. That is a reasonable concern, and the paper honestly labels Eq. (20) a 'budget-preserving surrogate rather than an exact second-order solution.' However, it is not the most decisive issue for the central claim, because the same-radius image-classification experiments (Table I) provide direct evidence that the allocation rule helps at fixed ρ. The most decisive issue is the radius mismatch in the transfer-learning section: comparing GEAR-SAM at ρ=0.2 against SAM at ρ=0.05 means the reported transfer gains cannot be attributed to GEAR-SAM's allocation mechanism. Because the reader already flagged this exact confound in the rationale and issued a CONDITIONAL verdict asking for matched-radius baselines, my independent read does not change the verdict. I do not fully agree with the reader's weighting: the theoretical surrogate issue is not load-bearing for the empirical claim, whereas the transfer confound is. The verdict remains CONDITIONAL, pending the matched-radius test, error bars for label-noise results, and released code/data.","tokens_in":14872,"tokens_out":3999,"duration_ms":38042,"concrete_test":"Rerun the transfer-learning benchmark (Table III) on EfficientNet-B0 and ResNet-50 for Flowers102 and Stanford Cars, holding all hyperparameters fixed, with four conditions: SAM ρ=0.05, SAM ρ=0.2, GEAR-SAM ρ=0.05, and GEAR-SAM ρ=0.2. Use at least 3 seeds and report mean ± std. If SAM ρ=0.2 matches or exceeds GEAR-SAM ρ=0.2, the transfer gains are attributable to the larger global radius, not to block-wise allocation; if GEAR-SAM still wins at matched radius, the confound is cleared.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that GEAR-SAM improves generalization 'across architectures and tasks' rests substantially on the transfer-learning results in Table III. But Section IV-D sets the perturbation radius of SAM to 0.05 and the GEAR-SAM radius to 0.2. In squared budget terms, GEAR-SAM is allowed 16× the SAM budget (0.04 vs. 0.0025). The paper elsewhere emphasizes that GEAR-SAM 'preserves the global SAM budget,' but in this comparison it does not preserve the baseline's budget. If larger perturbation radii are beneficial during fine-tuning — a plausible effect, especially for pretrained backbones — then the large transfer gains on Flowers102, Stanford Cars, and Oxford-IIIT Pet could be entirely a radius effect. No matched-radius baseline is reported. This is a direct empirical confound for a stated pillar of the central claim, independent of whether the EMA surrogate in Eq. (20) is theoretically justified. The label-noise table also lacks error bars and no code is provided, but the radius mismatch is the single most load-bearing issue because it invalidates a whole experiment family as evidence for the allocation mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GEAR-SAM, a modification of Sharpness-Aware Minimization (SAM) that maintains an exponential moving average (EMA) of squared block-gradient norms and uses this signal to allocate a fixed global perturbation radius across parameter blocks through the closed-form solution in Eq. (21). The authors prove that standard SAM implicitly performs a block-wise radius allocation, derive an exact identity for the minibatch gradient second moment (Eq. (15)), and present a budget-preserving allocation rule (Eq. (20)). Experiments cover CIFAR-10/100 image classification, transfer learning on five datasets, label-noise robustness, partition studies, and Hessian spectra, with the stated conclusion that GEAR-SAM improves generalization and robustness across architectures and tasks.","tokens_in":15176,"tokens_out":6065,"duration_ms":54842,"significance":"The paper's derivations are mathematically sound and its main CIFAR-100 results, which use matched perturbation radii, show consistent gains over SAM (0.66, 0.82, and 0.84 points on ResNet-18, WideResNet-28-10, and PyramidNet-110, respectively) with small standard deviations. The method is lightweight, preserves the global SAM budget by construction, and the Hessian top-eigenvalue measurement provides direct evidence of flatter minima. However, the transfer-learning comparison is confounded by unequal perturbation radii, the label-noise table lacks repeated runs, and the central allocation surrogate in Eq. (20) is only loosely connected to the second-order objective it is supposed to approximate. These issues currently prevent the paper from fully supporting its cross-task claims.","major_comments":[{"comment":"The transfer-learning experiments use a perturbation radius of 0.05 for SAM and 0.2 for GEAR-SAM. In squared radius, this is a 16x budget difference (0.04 vs. 0.0025). The large gains on Flowers102 (78.04 to 84.04), Stanford Cars (75.75 to 80.54), and Oxford-IIIT Pet (88.30 to 91.91) could therefore be entirely a global-radius effect rather than an effect of the block-wise allocation mechanism. Since the paper claims GEAR-SAM 'preserves the global SAM budget' and the transfer results are a stated pillar of the 'across architectures and tasks' claim, a matched-budget comparison is required. Please report SAM with rho=0.2 and/or GEAR-SAM with rho=0.05, and ideally a small sweep of rho for both methods.","section":"IV-D (Table III)"},{"comment":"The label-noise robustness table reports single-run accuracies with no standard deviations, despite the image-classification tables reporting three seeds. Moreover, SAM's CIFAR-10 accuracy at 80% noise (76.94) is higher than its accuracy at 60% noise (67.22), which is non-monotonic and unusual for symmetric label noise; this calls for explanation or suggests an unstable evaluation. Since robustness is one of the paper's stated contributions, at least three repeated runs per configuration (or another accepted uncertainty measure) and a discussion of the non-monotonicity are needed before this evidence can support the claim.","section":"IV-C (Table II)"},{"comment":"The allocation rule is the solution of a linear surrogate (maximize sum s_b r_b subject to sum r_b^2 <= rho^2), not of the second-order sharpness objective in Eq. (12). The paper explicitly acknowledges this, but the central mechanism -- that EMA squared-gradient energy is a curvature-related sensitivity signal -- is not directly validated. The Hessian spectrum in Fig. 2 is a useful global check, but it does not show that the allocated budget shares track block-wise curvature. Please add an analysis connecting the allocated radii to block-wise curvature (e.g., diagonal Hessian trace or directional curvature per block) or ablations showing that this particular surrogate is preferable to plausible alternatives. Without that, the method's explanation remains a heuristic alongside the empirical results.","section":"III-D (Eq. (20))"}],"minor_comments":[{"comment":"The paper says GEAR-SAM introduces 'no additional block-wise hyperparameters' and only scalar state; this is technically true, but the EMA factor beta and the stability constant delta are still additional global hyperparameters selected by tuning. Please state this explicitly to avoid understating the tuning burden.","section":"III-A and IV-E"},{"comment":"Many baseline numbers are taken directly from the BSAM paper [9]. Please state whether those baselines were run under the same learning rate, weight decay, and epoch schedule as GEAR-SAM; differences in protocols can materially affect the comparisons.","section":"IV-A (Table I)"},{"comment":"The statement that the common factor m cancels after block normalization is correct but brief; a reader may wonder why the factor does not affect the normalized allocation. One clarifying sentence would help.","section":"Appendix / Eq. (15)"},{"comment":"No code or reproducibility statement is provided. For a method whose contribution is an algorithmic rule, making the implementation available (or at least specifying the exact block partition for each architecture) would significantly strengthen the paper.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The CIFAR-100 classification results are the strongest part of the paper and appear credible with matched radii. The transfer and label-noise sections are the main barriers: the transfer radius mismatch should be fixed before publication, and the label-noise table needs repeated runs. I would also flag that a substantial number of baseline numbers come from a single prior paper [9] by the same group, and independent reproduction or additional baselines would increase confidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"GEAR-SAM is a modest but sensible addition to the SAM literature. The core idea is simple: instead of allocating SAM's perturbation budget by instantaneous gradient norms, use an EMA of squared block gradients as a sensitivity score and allocate radii by Cauchy-Schwarz. The math is correct — Observation 1 is just Pythagoras, Proposition 1 follows from Cauchy-Schwarz, and Eq. (15) is exact. The method adds no Hessian-vector products and only B scalar states, so the complexity claim is honest. I think the CIFAR-100 results are the strongest evidence: with matched ρ=0.2, GEAR-SAM beats SAM by 0.66–0.84 points across three architectures, and the Hessian analysis shows lower maximum eigenvalue.\n\nThe weak spot is the transfer-learning section, and it's not minor. In Table III, SAM is run with ρ=0.05 while GEAR-SAM uses ρ=0.2 — a 16x difference in squared budget. The paper stresses that GEAR-SAM 'preserves the global SAM budget,' but in that table it does not preserve the baseline's budget. The large gains on Flowers102, Stanford Cars, and Oxford-IIIT Pet (up to 6 points) could easily be a radius effect, not an allocation effect. The authors need to run GEAR-SAM at ρ=0.05 and, ideally, SAM at ρ=0.2 as well. Without that, the 'across tasks' claim collapses to the CIFAR-100 results.\n\nOther issues are smaller: label-noise results have no error bars, and no code is provided. The derivation is admittedly a surrogate — the paper says Eq. (20) is 'a budget-preserving surrogate rather than an exact second-order solution' — so the empirical results are doing the work, which makes the confound more costly. The related work is adequately cited, and the self-citation to BSAM is for baseline numbers, not central.\n\nBottom line: this is a legitimate, incremental method with a correctly derived allocation rule and a solid classification experiment. The transfer experiment as presented is invalid evidence for the mechanism, and that needs fixing before the paper is publishable. I'd send it to peer review — a good referee would catch this and the authors can address it with a few additional runs. I wouldn't cite it in its current form.","headline":"Smart, cheap SAM variant with correct math and good CIFAR-100 results; transfer gains are inflated by an unmatched perturbation radius.","tokens_in":15619,"tokens_out":2110,"would_cite":false,"duration_ms":16893,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GEAR-SAM reallocates SAM's fixed perturbation budget by historical gradient energy and beats SAM by up to 0.84 on CIFAR-100.","keywords":["Sharpness-Aware Minimization","block-wise perturbation","gradient energy","exponential moving average","budget allocation","flat minima","label noise robustness","Hessian spectrum"],"falsifier":"Measure, on a training run, the directional block curvatures q_b = u_b^T H_bb u_b at the perturbed directions and correlate them with the EMA scores s_b used by GEAR-SAM; if blocks that receive more budget do not have systematically higher curvature — or if a run with the allocation rule reversed (more budget to low-energy blocks) still matches GEAR-SAM's accuracy — then the proposed mechanism, not the general idea of redistribution, is what needs revision.","tokens_in":14757,"feed_emoji":"🎯","tokens_out":6805,"duration_ms":55973,"temperature":0.7,"pith_summary":"The paper argues that Sharpness-Aware Minimization (SAM) already performs an implicit, block-wise allocation of its perturbation budget, but it does so using only the current minibatch gradient, which is noisy and ignores how each network block's sensitivity changes over training. GEAR-SAM replaces that instantaneous signal with an exponential moving average of squared block-gradient norms, treating this 'gradient energy' as a cheap curvature-related proxy, and derives a closed-form rule that reallocates the same global radius across blocks. The result is a perturbation of no larger norm than SAM's that reaches flatter minima: on CIFAR-100, GEAR-SAM improves accuracy over SAM by 0.66, 0.82, and 0.84 points on ResNet-18, WideResNet-28-10, and PyramidNet-110, and it lowers the largest Hessian eigenvalue found by SAM. The broader claim is that a fixed sharpness-perturbation budget should be dynamically redistributed as functional blocks' roles evolve, not kept at fixed or purely instantaneous shares.","feed_headline":"Reallocating SAM's radius by gradient history: +0.84 on CIFAR-100","feed_subtitle":"Fixed SAM radius, reallocated by gradient history, improves sharpness and noise robustness at no extra cost.","key_machinery":"The load-bearing object is the block-wise gradient-energy score s_{b,t}, the EMA of squared ℓ₂ norms of the gradient restricted to each functional block. From these scores the allocation rule r*_b = ρ s_b / sqrt(Σ_j s_j²) (Proposition 1) distributes the fixed SAM radius ρ. The rule is the exact maximizer of the linear alignment surrogate Σ_b s_b r_b subject to Σ r_b² ≤ ρ², proved by Cauchy–Schwarz; because EMA is linear, the score is equivalent to summing adaptive-gradient-style per-parameter second moments within a block. The architecture-aware partition (stem, coarse residual stages, classifier; normalization affine parameters kept with their block) supplies the blocks and stabilizes the s","core_discovery":"The paper sets out to establish that the practical SAM perturbation, while optimal for the first-order sharpness objective, ignores curvature-related block sensitivity because it allocates each block a radius proportional to its instantaneous gradient norm. GEAR-SAM instead maintains B scalar scores — an exponential moving average of squared block-gradient norms — and allocates the fixed SAM radius by r_b = ρ s_b / sqrt(Σ_j s_j²), which uniquely maximizes the linear surrogate Σ_b s_b r_b under the budget constraint Σ r_b² ≤ ρ². This preserves the global SAM budget, never enlarges the SAM neighborhood, requires no Hessian-vector products or explicit Fisher estimation, and adds only B scalar s","pith_inferences":["A testable corollary the authors leave implicit: if the mechanism is correct, GEAR-SAM's advantage over SAM should track the heterogeneity of block sensitivities — datasets or architectures where all blocks have similar gradient energy should show near-zero gains, while those with skewed energy should show large gains. That is directly measurable from the EMA scores.","The same budget-redistribution principle could be combined with other perturbation schemes, such as direction-changing or layer-wise-scaled variants; nothing in the derivation forces the in-block direction to be the raw gradient, so a natural extension is to use GEAR-SAM's radii with learned or adaptive directions.","Because the score is an EMA, the method is a continuous, soft analogue of sparse or selective perturbation methods: instead of deciding whether to perturb a block, it lets blocks fade in and out of the budget. This suggests a calibration test — comparing GEAR-SAM's block share trajectories with the pruning importance of blocks — to see whether the energy signal is tracking representational importa","The Hessian spectrum result predicts the benefit shows up under worst-case perturbations, not average-case curvature; a concrete check is adversarial robustness or sharpness under random directions, where GEAR-SAM should show larger margins relative to SAM."],"forward_implications":["Because the total perturbation norm is never larger than SAM's, any accuracy gain comes purely from where the budget is spent, not from a stronger sharpness penalty; GEAR-SAM can be applied as a drop-in replacement for SAM with no extra forward-backward passes.","The method reduces worst-case local sharpness: on ResNet-18/CIFAR-100 the maximum Hessian eigenvalue drops from 200.72 (SAM) to 192.92, while the top-50 mean stays roughly comparable.","The gains concentrate on harder, finer-grained tasks: CIFAR-100 gains are 0.66–0.84 points over SAM while CIFAR-10 differences are near zero, suggesting historical sensitivity matters most when representations must support finer discrimination.","Under symmetric label noise, GEAR-SAM outperforms SAM at all four tested noise rates on CIFAR-10 and at 40%, 60%, and 80% on CIFAR-100, with particularly large margins at high noise.","The benefit is not tied to one partition: coarse, fine, and tensor-wise groupings all beat SAM on CIFAR-100, with the default coarse blocks nearly matching the best fine-grained split."],"fun_headline_variants":["GEAR-SAM reallocates SAM's radius via gradient history","EMA of block gradients guides SAM budget: +0.84 on CIFAR-100","Redistribute SAM perturbation by sensitivity, keep radius fixed","Gradient-history radius beats norm-based allocation","SAM budget reallocated by block sensitivity, no Hessian needed"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that an EMA of squared block-gradient norms tracks the block-level curvature that matters for flatness, and that the linear alignment rule in Eq. (20) is a good way to spend the fixed SAM budget; the paper itself labels Eq. (20) a budget-preserving surrogate, not an exact second-order solution.","fun_headline_variants_meta":{"raw":{"variants":["GEAR-SAM reallocates SAM's radius via gradient history","EMA of block gradients guides SAM budget: +0.84 on CIFAR-100","Redistribute SAM perturbation by sensitivity, keep radius fixed","Gradient-history radius beats norm-based allocation","SAM budget reallocated by block sensitivity, no Hessian needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000262,"raw_usage":{"total_tokens":1419,"prompt_tokens":714,"completion_tokens":705,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":616}},"tokens_in":458,"tokens_out":705,"duration_ms":7900,"temperature":1.0,"reasoning_tokens":616,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T22:33:19.443156+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, on a training run, the directional block curvatures q_b = u_b^T H_bb u_b at the perturbed directions and correlate them with the EMA scores s_b used by GEAR-SAM; if blocks that receive more budget do not have systematically higher curvature — or if a run with the allocation rule reversed (more budget to low-energy blocks) still matches GEAR-SAM's accuracy — then the proposed mechanism, not the general idea of redistribution, is what needs revision.","supporting_citations":[],"review_version":1}