{"id":"c287f742-d1bb-4de9-a6be-a2c667bd3384","arxiv_id":"2505.01168","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"HEAT improves black-box adversarial transferability by combining SVD-derived shared gradient directions with adaptive per-model weights inspired by domain generalization.","lead":"This paper proposes HEAT, an ensemble attack that uses singular value decomposition to find a common gradient direction and adaptive weights to combine surrogate models, reporting higher black-box attack success rates on CIFAR-10, CIFAR-100, and ImageNet. A generalist would read it to see a new design pattern for transferable adversarial attacks, though the gains currently come without code, error bars, or a complete algorithm.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"C-GRADS may be selecting the largest-norm model's gradient rather than a consensus direction, because Eq. 4 stacks unnormalized gradients before SVD; this threatens the central novelty and should be tested by row normalization.","rationale":"The reader's weakest assumption was that the top-k singular subspace of G contains the transferable shared directions. My concern sharpens this: because the rows of G are raw, unnormalized gradients, the SVD is not a consensus operator but a variance-maximizing one, so the leading singular vectors may simply encode the largest-gradient model. This is a concrete, testable flaw in the construction, not a matter of tuning p. It is load-bearing because the paper's core novelty is the consensus mechanism; if row normalization eliminates the gains, the method is not doing what is claimed. I do not move the verdict to REJECT because the empirical results could still hold for other reasons (e.g., magnitude weighting is a valid heuristic), and the reader's CONDITIONAL verdict already requires code and error bars. My concern adds a specific experiment that should be part of the acceptance conditions, but it does not by itself overturn the conditional assessment. Hence UNCHANGED, with the condition sharpened.","tokens_in":12430,"tokens_out":7741,"duration_ms":81351,"concrete_test":"Re-run the CIFAR-10 and ImageNet experiments with two variants: (i) row-normalize each gradient in Eq. 4 before SVD, and (ii) set k=1 and k=M in Eq. 6/7 while keeping p=0.7 for comparison. Also compare against simply using the gradient of the surrogate with the largest norm. If HEAT's ASR advantage over Ens persists after row normalization and is insensitive to k, the consensus mechanism is plausible; if performance collapses or tracks the max-norm model, the C-GRADS mechanism is not doing consensus work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equations 4-7 define C-GRADS by building a gradient matrix G from raw model gradients and taking a singular-value-weighted sum of the top-k right singular vectors. Because the rows of G are not normalized, the singular value decomposition is scale-dependent: a model with a large gradient norm will dominate the leading singular vectors. Thus V_k in Eq. 7 is not necessarily a direction shared by all models; it can be nearly identical to the gradient of the single strongest model. The paper's central claim that C-GRADS synthesizes consensus gradient directions that transfer to black-box models is therefore not established by the construction as written. If the reported improvements over Ens are largely due to this implicit magnitude weighting, the method reduces to a heuristic that happens to emphasize one surrogate, and the stated domain-generalization mechanism is not the cause. This is load-bearing because the paper's novelty and the 'shared direction' explanation both rest on this step, and because the ablation in Table 3 does not separate consensus from magnitude effects.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes HEAT, an ensemble adversarial attack that aims to improve black-box transferability by combining two modules: C-GRADS, which performs SVD on a matrix of white-box surrogate gradients and uses a weighted sum of leading right-singular vectors as a 'shared' attack direction, and D-HARMO, which reweights each surrogate gradient using an intra-domain coherence term and an inter-domain divergence/alignment term. The reported experiments on CIFAR-10, CIFAR-100, and ImageNet against eight black-box architectures show average attack success rate gains over the Ens baseline, with particularly large improvements on vision transformers, and an ablation study attributes the gains to the proposed components.","tokens_in":12621,"tokens_out":9137,"duration_ms":83873,"significance":"If the results hold, HEAT provides a simple and effective recipe for ensemble attacks: SVD-based direction synthesis plus adaptive gradient weighting yields state-of-the-art transferability on CNNs and ViTs, and it integrates naturally with I-FGSM, MI-FGSM, and DI2-FGSM. The paper should be credited for the breadth of its evaluation (three datasets, eight black-box models, multiple base attacks) and for including an ablation study. However, the claimed 'first formal connection' to domain generalization is not formalized, and the lack of code, error bars, and sensitivity analyses means the empirical claims are not yet fully supported.","major_comments":[{"comment":"The prose states that 'Models with higher gradient similarity (i.e., greater alignment) are assigned larger alignment contribution factors,' but Eq. (16) defines A_m as the reciprocal of the mean cosine similarity, so higher P_mj produces a smaller A_m. Because A_m enters the entropy H_m (Eq. 19) and hence the inter-domain weights, this contradiction changes which models are emphasized. Please correct the equation or the prose and rerun the affected experiments and ablations.","section":"Section 3.2.2, Eq. (16)"},{"comment":"G is built from unnormalized gradients, so the leading singular vectors are scale-dependent: a surrogate with a large gradient norm can dominate the top singular subspace, and V_k may approximate that model's gradient rather than a shared direction. The central claim that C-GRADS synthesizes consensus directions is therefore not established as written. Please add an experiment with row-normalized gradients (or another scale-invariant variant) and report whether the gains persist; otherwise the improvement over Ens may be an implicit magnitude reweighting rather than consensus extraction.","section":"Section 3.2.1, Eqs. (4)-(7)"},{"comment":"The number of retained singular vectors k is determined by the hand-set p=0.7, but no ablation over p is reported, so the sensitivity of the headline results to this key hyperparameter is unknown. In addition, the text calls the ratio a 'contribution to total variance' while using raw singular values; with SVD the variance interpretation applies to squared singular values. Please either report an ablation over p and justify the variance terminology, or use squared singular values.","section":"Section 3.2.1, Eq. (6)"},{"comment":"The text describes an iterative process and the base attacks I-FGSM, MI-FGSM, and DI2-FGSM are iterative, but Algorithm 1 shows a single forward pass ending in one update x*_adv = Clip(x + alpha sign(g)), with no loop over iterations and no indication of how C-GRADS and D-HARMO are reused in each iteration. This makes the method ambiguous and hard to reproduce. Please present the full iterative procedure (e.g., with x^{i-1}_{adv} on the right-hand side of Eqs. 8 and 24) and specify which quantities are recomputed per iteration.","section":"Section 3, Algorithm 1 and Eqs. (8)/(24)"},{"comment":"The quantitative claims in the text are not consistent with the tables. The sentence 'HEAT achieves ASR improvements of 5.11%, 22.76%, and 27.22%' for I-FGSM, MI-FGSM, and DI2-FGSM does not match Table 2 (for I-FGSM, HEAT is 45.42 vs Ens 16.87, an absolute difference of 28.55 points), and Table 1 contains several cells where HEAT is below AdaEA (e.g., CIFAR-100 BiT-101: 46.90 vs 55.38), contradicting 'consistently outperforms.' Please restate these claims precisely, define the metric used for 'improvement,' and add error bars or significance tests over multiple runs.","section":"Section 4.2"}],"minor_comments":[{"comment":"The two rows marked '✓ ✓ ✓' are not distinguished in the header; please label the component sets explicitly (e.g., A+B+C, A+B+D, A+B+C+D).","section":"Table 3"},{"comment":"The temperature parameter τ appears in Algorithm 1 and Eqs. (17)-(18) but its value is not reported; please provide it.","section":"Section 4.1"},{"comment":"The SVD sign ambiguity is not addressed; since v_i are defined up to sign, the sum V_k in Eq. (7) may flip direction. Please specify a sign convention (e.g., align each v_i with the mean gradient).","section":"Section 3.2.1"},{"comment":"The intra-domain weight in Eq. (11) can be negative when the adversarial example of model m does not transfer to model j; after normalization this can produce negative weights, which is not discussed.","section":"Section 3.2.2"},{"comment":"There are several typos: 'Introdution' in Section 1, 'HEA T' in the introduction, 'D-HarMony' in Section 3.2, and 'Comparision' in Section 4.1. Figure 2 also spells the module as 'C-GARDS' in one place.","section":"Section 1"},{"comment":"The claim of a 'first formal connection between domain generalization and adversarial attacks' is not backed by any formal statement in the paper; please soften it or add a formal treatment.","section":"Sections 1 and 5"},{"comment":"No code is provided; given the number of hyperparameters (p, τ, ε) and the ambiguity in Algorithm 1, releasing code would materially improve reproducibility.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like an unfinished submission: the Eq. (16) contradiction, the missing iteration loop in Algorithm 1, and the numeric mismatches in Section 4.2 suggest the final check was rushed. The empirical scope is good and the core idea is worth pursuing, but the authors should be asked to supply code, sensitivity analyses, and corrected claims before the results can be trusted. I do not see evidence of misconduct, but the 'first formal connection' framing is overstated relative to what is actually shown."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [Name],\n\nQuick take on arXiv:2505.01168. It's a heuristic ensemble attack that reports very large gains over existing ensemble baselines on three datasets, across CNNs and ViTs. The ingredients are known (SVD, cosine similarity, entropy weighting, adaptive fusion) but the specific combination is new: C-GRADS uses top-k singular vectors of a stacked gradient matrix to define a shared direction, and D-HARMO blends two adaptive weights (intra-model coherence via cross-predicted losses, inter-model divergence via loss and alignment factors). If the numbers hold, it's a solid incremental advance in black-box attack transferability.\n\nThe tables look good and the ablation supports each component helping. But there are several soft spots that prevent me from trusting the headline claim as written. No code, no error bars, no significance tests, and no sensitivity analysis for the key hyperparameter p (fixed at 0.7). More seriously, Eq. 16 is the reciprocal of the mean cosine similarity, so models with lower alignment get larger factors, contradicting the prose and the intuition in the ablation. Algorithm 1 shows a single update, not the iterative loop described in Section 3.1.\n\nThe bigger conceptual concern, raised by a stress-test, is that C-GRADS stacks raw gradients without row normalization before SVD. SVD is scale-dependent, so the leading singular vectors can be dominated by the single model with the largest gradient norm. The 'shared consensus direction' may therefore be nothing more than that model's gradient, and the reported gains over Ens might come from implicit magnitude weighting rather than a genuine consensus. This isn't tested at all; the ablation doesn't separate scale from consensus. The authors should normalize rows or otherwise ablate this.\n\nThat said, the paper is not circular: the target metrics are held-out black-box models, and p is set by hand rather than fitted.\n\nOverall, it's a plausible and potentially useful heuristic, but the evidence as presented is insufficient. I'd send it to peer review, not desk-reject it, because the idea is real and the flaws are addressable. A good referee would ask for code, row-normalization experiments, a corrected Eq. 16, and a proper iterative algorithm.\n\nWorth a skim if you work on adversarial robustness; I wouldn't cite it yet.\n\nBest.","headline":"Plausible ensemble attack with impressive tables, but the central 'consensus gradient' mechanism is undercut by an untested SVD scale issue and a contradictory formula; deserves a careful referee rather than rejection.","tokens_in":13162,"tokens_out":3661,"would_cite":false,"duration_ms":35669,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HEAT synthesizes shared gradient directions with SVD and dynamically weights surrogate models to boost adversarial transferability.","keywords":["adversarial transferability","ensemble attack","singular value decomposition","domain generalization","black-box attack","adaptive weighting","gradient alignment"],"falsifier":"Run HEAT while sweeping the cumulative variance ratio $p$ from small values to $p=1$ (keeping all singular directions) on the same surrogate and target models. If attack success rate does not drop sharply when $p$ is reduced well below 0.7, or if $p=1$ matches or beats $p=0.7$, the claim that the top-$k$ consensus subspace carries the transferable signal is falsified; a second check is to repeat with ensembles whose gradients are intentionally made diverse (e.g., models trained on disjoint data), where the shared-subspace assumption should break down.","tokens_in":12209,"feed_emoji":"🎯","tokens_out":4443,"duration_ms":39636,"temperature":0.7,"pith_summary":"This paper proposes HEAT, an ensemble attack that crafts adversarial examples with much higher black-box transferability by treating the surrogate models as a source domain and the unseen targets as a target domain. It claims that averaging gradients across models is suboptimal because shared, transferable signal is diluted by model-specific noise. HEAT replaces simple averaging with two modules: C-GRADS, which applies SVD to the stacked gradient matrix and keeps only the top $k$ singular directions, and D-HARMO, which assigns each model a dynamic weight from intra-domain coherence and inter-domain divergence. On CIFAR-10, CIFAR-100, and ImageNet, the paper reports that HEAT outperforms Ens, SVRE, and AdaEA on CNN and ViT black-box models, with an average attack success rate improvement over the Ens baseline exceeding 28%.","feed_headline":"Ensemble attack gains 28% more black-box transferability","feed_subtitle":"SVD and adaptive weighting find shared gradient directions across surrogate models.","key_machinery":"The central object is the consensus gradient direction $V_k$, produced by C-GRADS via SVD: gradients from $M$ surrogate models are stacked row-wise into $G\\in\\mathbb{R}^{M\\times D}$, and the right-singular vectors weighted by their singular values give a shared perturbation direction. The second mechanism is D-HARMO's dual weight: the intra-domain weight $w^{\\mathrm{intra}}_m$ compares how much model $m$'s own adversarial step raises other models' losses, and the inter-domain weight $w^{\\mathrm{inter}}_m$ combines a normalized loss-contribution factor with gradient alignment (cosine similarity) into an information-entropy score; the final gradient is $\\sum_m w^{\\mathrm{intra}}_m w^{\\mathrm{inter}}_m \\nabla_x L_m(x^{V_k}_{\\mathrm{adv}})$. These two machinery pieces are what carry the claim that shared directions survive projection while per-model noise is weighted down.","core_discovery":"HEAT's central claim is that the transferable component of an ensemble of surrogate gradients lies in a low-dimensional consensus subspace. Concretely, the paper constructs the gradient matrix $G$ by stacking per-model gradients and decomposes it as $G = U\\Sigma V^\\top$; it then keeps the $k$ singular directions that account for fraction $p=0.7$ of total variance and forms a shared direction $V_k$ as a singular-value-weighted sum of those top-$k$ vectors. Around this shared direction, D-HARMO computes per-model weights: intra-domain coherence rewards models whose gradient direction, when applied alone, also raises the loss of other models, while inter-domain divergence rewards models with large loss contribution and gradient alignment with the ensemble. The final perturbation updates along the sign of the weighted sum of model gradients at the C-GRADS example. The paper asserts that this combination yields substantially higher attack success rates against eight unseen CNN and ViT models across three datasets, and that each component contributes in ablation.","pith_inferences":["The paper fixes the SVD retention ratio at $p=0.7$; an immediate extension would sweep $p$ per ensemble and per dataset, since the optimal subspace rank should depend on how diverse the surrogate models are.","Because the method's success depends on shared singular directions, a defense could try to detect or destroy adversarial perturbations by projecting them onto the top-$k$ subspace of the input's gradient ensemble, if gradient information is available.","The intra-domain weighting resembles a leave-one-model-out generalization estimator; a natural test is whether the same weights predict which single model transfers best, which would turn D-HARMO into a model-selection tool."],"forward_implications":["If the consensus subspace really captures transferable signal, then HEAT should raise black-box attack success rate on held-out CNN and ViT targets, which the paper reports across CIFAR-10, CIFAR-100, and ImageNet.","Because D-HARMO adapts weights per input and per iteration, HEAT should remain effective when the ensemble is heterogeneous (CNNs plus ViTs), not only homogeneous.","The SVD projection plus entropy weighting should compose with base attack methods such as I-FGSM, MI-FGSM, and DI2-FGSM, which the paper confirms.","If both modules are load-bearing, removing either should degrade attack success rate; the ablation shows C-GRADS contributes the largest gain, especially against ViT targets."],"supporting_citations":[{"why":"Defines the Ens baseline whose simple gradient averaging HEAT replaces and claims to outperform.","marker":"(Liu et al., 2022)"},{"why":"SVRE baseline that HEAT compares against and aims to beat in transferability.","marker":"(Xiong et al., 2022)"},{"why":"AdaEA baseline, the adaptive ensemble attack that HEAT must surpass.","marker":"(Chen et al., 2023a)"},{"why":"Provides the low-rank approximation result used by C-GRADS to justify keeping only the top singular directions.","marker":"(Eckart & Young, 1936)"}],"fun_headline_variants":["SVD consensus boosts ensemble attack transferability","HEAT: harmonized ensemble attacks transfer better","28% better transfer via SVD consensus gradients","Gradient harmony lifts adversarial transfer across models","Adaptive weights sharpen ensemble attack transfer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's gains depend on the assumption that the transferable gradient signal of the surrogate ensemble is concentrated in the top few singular directions (captured at $p=0.7$), so that throwing away the remaining directions removes noise rather than useful signal.","fun_headline_variants_meta":{"raw":{"variants":["SVD consensus boosts ensemble attack transferability","HEAT: harmonized ensemble attacks transfer better","28% better transfer via SVD consensus gradients","Gradient harmony lifts adversarial transfer across models","Adaptive weights sharpen ensemble attack transfer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000247,"raw_usage":{"total_tokens":1520,"prompt_tokens":902,"completion_tokens":618,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":551}},"tokens_in":518,"tokens_out":618,"duration_ms":5899,"temperature":1.0,"reasoning_tokens":551,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:24:29.681708+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run HEAT while sweeping the cumulative variance ratio $p$ from small values to $p=1$ (keeping all singular directions) on the same surrogate and target models. If attack success rate does not drop sharply when $p$ is reduced well below 0.7, or if $p=1$ matches or beats $p=0.7$, the claim that the top-$k$ consensus subspace carries the transferable signal is falsified; a second check is to repeat with ensembles whose gradients are intentionally made diverse (e.g., models trained on disjoint data), where the shared-subspace assumption should break down.","supporting_citations":[{"cited_title":"Delving into transferable adversarial examples and black-box attacks","cited_arxiv_id":null,"evidence_quote":"Defines the Ens baseline whose simple gradient averaging HEAT replaces and claims to outperform."},{"cited_title":"E., and He, K","cited_arxiv_id":null,"evidence_quote":"SVRE baseline that HEAT compares against and aims to beat in transferability."}],"review_version":1}