{"id":"11735f8c-2b5e-4a82-b8ef-59d121ec4d43","arxiv_id":"2508.12384","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ViT-EnsembleAttack augments each ViT surrogate with three randomized strategies, tunes their parameters by Bayesian optimization, and ensembles them to substantially improve adversarial transferability.","lead":"The paper introduces ViT-EnsembleAttack, a method that augments vision transformer surrogate models with random modifications (head dropping, attention scaling, MLP mixing) to generate adversarial examples that transfer better to unknown models. On eight CNN and eight ViT targets it reports large gains over existing ensemble attacks, and the code is released.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 2 never projects onto the L∞ ball; with α=4.8 and T=10, perturbations can reach norm 48 vs the stated ε=16, so the reported margins may reflect out-of-budget attacks unless clipping is explicitly applied.","rationale":"The strongest claim is an empirical one: a large, consistent ASR gain over state-of-the-art ensemble attacks. That claim is only meaningful if all methods operate under the same ε=16 budget. The paper's Section 3.1 defines the attack as a constrained problem, but Algorithm 2 does not implement the constraint. The arithmetic is simple: with α=q·ε/T=4.8 and T=10, the worst-case L∞ norm is 48, so the printed method is not generating adversarial examples within the stated budget. This is not a subtle modeling choice; it violates the threat model used by every baseline and by the paper's own definition. If the authors' implementation clips, as many implementations do, then the paper is merely under-specified; if it does not, the headline margins are inflated. Either way the manuscript needs a correction, and the availability of the code repository makes this check feasible. I also note a second reproducibility defect: the inner loop in Algorithm 2 (lines 13–16) assigns L_i but does not accumulate over `loop`, even though Fig. 3(b) shows a clear effect of loop; the pseudocode appears to omit the gradient accumulation that the experiments rely on. That issue is secondary, however, because the budget violation directly undermines the quantitative claim. My recommendation is unchanged from the reader's CONDITIONAL verdict: the central idea is plausible and the ablations are encouraging, but the manuscript should be revised or verified only after the projection and step-size control issue is resolved and the code is checked.","tokens_in":17773,"tokens_out":7094,"duration_ms":77170,"concrete_test":"Re-run the MI-FGSM rows of Tables 1–2 with an explicit projection onto ‖δ‖∞≤16 after every update, and also run baselines with the enlarged step size α=4.8 under the same projection. If the average margin over SMER drops materially, the printed algorithm's missing projection was inflating the result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the comparisons in Tables 1–2 being made at a fixed perturbation budget ε=16. Algorithm 2 (line 21) updates x_{t+1} = x_t + α·sign(g_{t+1}) and never clips x_{t+1} or δ_{t+1} back to the ε-ball. With the chosen hyper-parameters (§4.1), α = q·ε/T = 3×16/10 = 4.8. Over T=10 iterations, the L∞ norm of the final perturbation can be as large as 48, three times the stated bound. Even if sign changes reduce the realized norm on many images, nothing in the printed algorithm guarantees ‖δ‖∞≤16; therefore the attack is not the constrained optimization problem defined in Eq. (1). Baselines use α=1.6 and hence cannot exceed 16 even without projection, so the reported 15.3% average gain over SMER may be an artifact of a larger effective perturbation budget. If the released code does clip, the paper should state this in Algorithm 2; the present pseudocode is insufficient to support the correctness of the headline result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes ViT-EnsembleAttack, an ensemble-based adversarial attack for Vision Transformers. Instead of only reweighting fixed surrogate models, the method augments each surrogate with three stochastic structural perturbations: Multi-head dropping (MHD), Attention score scaling (ASS), and MLP feature mixing (MFM). The augmentation parameters are tuned by Bayesian optimization using transfer performance to the other surrogate models, and the final attack ensembles the augmented variants with an automatic loss reweighting and an enlarged step size. Experiments on 1,000 ImageNet images report attack success rates against eight ViT targets and eight CNN targets, with claimed average gains over the SMER baseline of roughly 4.6 percentage points on ViTs and 15.3 percentage points on CNNs, together with ablations, hyperparameter sensitivity studies, and computational cost comparisons.","tokens_in":17992,"tokens_out":4431,"duration_ms":49179,"significance":"If the reported margins survive a properly constrained rerun, the paper would make a useful contribution to the adversarial-transferability literature: it reframes ensemble attacks as a problem of generating diverse surrogate variants rather than only combining fixed models, and it provides three ViT-specific augmentation strategies with a Bayesian model-selection procedure. The evaluation is broad, covering normally trained, robust, and adversarially trained ViT and CNN targets, and the authors state that code is released, which are strengths. The Bayesian optimization protocol is not circular: parameters are selected on sibling surrogate models and final evaluation is on disjoint held-out targets, which is legitimate model selection. However, two load-bearing issues currently prevent acceptance: Algorithm 2 does not enforce the L-infinity budget stated in Eq. (1), and the stochastic pipeline is reported without seeds, error bars, or confidence intervals. The central quantitative claims are therefore not yet supported.","major_comments":[{"comment":"The update in Algorithm 2 (line 21), x_{t+1} = x_t + alpha * sign(g_{t+1}), does not project or clip the perturbation back to the L-infinity ball of radius epsilon. With the hyperparameters in Section 4.1, alpha = q * epsilon / T = 3 * 16 / 10 = 4.8, so over T = 10 iterations the unconstrained L-infinity norm can reach 48, three times the stated epsilon = 16. The baselines use alpha = 1.6 and therefore remain within budget even without projection, so the margins in Tables 1 and 2 may reflect a larger effective perturbation budget rather than genuinely stronger transferability. The authors must either add an explicit projection/clipping step and state it in the pseudocode, or rerun all comparisons at the same constrained budget. As printed, the attack does not solve the constrained optimization problem defined in Eq. (1).","section":"Algorithm 2, Section 4.1, Eq. (1)"},{"comment":"The attack is stochastic in several places: MHD/ASS/MFM draw random masks and scales at each forward pass, Bayesian optimization includes randomness, and the inference loop samples multiple paths. Yet the paper reports no random seeds, confidence intervals, error bars, or repeated-run statistics. Since the claimed margins over SMER are 4.6% on ViTs and 15.3% on CNNs, and the strongest baselines are themselves stochastic, the reader cannot judge whether these margins are statistically meaningful. Please report mean and standard deviation (or equivalent) over at least three seeds for the proposed method and the baselines in Tables 1-3 and Figure 4.","section":"Section 4.1, Tables 1-2"}],"minor_comments":[{"comment":"The text says random scaling factors follow a \"uniform contribution\"; this should presumably be \"uniform distribution.\"","section":"Section 3.3, Attention score scaling"},{"comment":"The pseudocode is unclear about how the chosen strategy c and parameter p are passed to the objective function OF; clarify the signatures of gp_minimize calls so the three optimization loops are unambiguous.","section":"Algorithm 2, lines 5-7"},{"comment":"The table header \"FLOPs (P)\" calls FLOPs \"floating-point operations per second,\" but the reported values appear to be total operations (peta-FLOPs); please correct the terminology.","section":"Table 4"},{"comment":"The caption states that computational cost grows \"exponentially\" with the inference loop count, but only a few discrete points are shown; either provide a precise scaling expression or soften the claim.","section":"Figure 3(d)"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue is the missing projection in Algorithm 2. I would ask the editor to require the authors to confirm whether the released code clips to epsilon = 16. If it does, the manuscript can likely be repaired with a corrected pseudocode and variance reporting; if it does not, the headline results are invalid under the stated threat model. The paper otherwise fits the scope of the journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things up front. First, the core idea — randomly augmenting ViT surrogates with multi-head dropping, attention score scaling, and MLP feature mixing, then tuning the randomness via Bayesian optimization — is a sensible extension of the ghost-network and input-transformation line of work. Second, Algorithm 2 never clips the perturbation back to the epsilon-ball. With q=3, alpha=4.8, T=10, the final perturbation can reach L-infinity norm 48 against the stated budget of 16. If the released code does clip, then this is a presentation bug, not a fatal flaw; if it doesn't, the reported 15.3% average gain over SMER is an artifact of an unconstrained budget. The paper needs to state explicitly whether clipping is applied and show the projection in the pseudocode.\n\nWhat the paper does well: the experimental setup is extensive — eight ViTs and eight CNNs, including robust and adversarially trained models — and the ablation study is thorough. The Bayesian optimization on sibling surrogates is legitimate model selection, not circular reasoning, and the authors correctly separate the tuning phase from the final evaluation. The automatic reweighting and step size enlargement modules are simple but well motivated, and the sensitivity analysis for q is honest.\n\nSoft spots that are real but secondary: no error bars or random seeds are reported for a stochastic attack, so we don't know whether the margins are stable. Step size enlargement is applied to the proposed method but not to the baselines, which makes the comparison less clean — adding it to SMER or SVRE would strengthen the claim that the gains come from augmentation rather than just a larger effective step. And the \"first tailored for ViTs\" claim would be more convincing with a direct comparison to ghost networks or attention-dropout baselines, which are closely related.\n\nOverall, I think the central mechanism is likely real and the work is a solid incremental contribution, but the projection omission in the pseudocode is load-bearing for the headline numbers. The paper deserves a serious referee, not a desk reject, so I'd send it to review with a clear request to fix the algorithm description and report variance. If the code clips, this becomes a moderately strong paper; if it doesn't, the empirical claims need to be re-run under the stated budget.","headline":"Plausible and useful ViT-specific ensemble augmentation with a serious presentation flaw: Algorithm 2 omits the L-infinity projection, which could inflate the headline gains unless the code clips.","tokens_in":18527,"tokens_out":3413,"would_cite":false,"duration_ms":36110,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adversarially augmenting the surrogate ViTs rather than reweighting their outputs is what unlocks stronger attack transferability.","keywords":["adversarial transferability","ensemble-based attacks","Vision Transformers","model augmentation","multi-head dropping","attention score scaling","MLP feature mixing","Bayesian optimization"],"falsifier":"Run the released code at the paper's settings and measure the $\\ell_\\infty$ norm of every final adversarial image; if many exceed 16, re-run the Table 1 and Table 2 comparisons with per-iteration projection onto $\\{\\delta:\\|\\delta\\|_\\infty\\le 16\\}$ and check whether the reported 4.6-point and 15.3-point margins survive.","tokens_in":17541,"feed_emoji":"🎯","tokens_out":10698,"duration_ms":100555,"temperature":0.7,"pith_summary":"The paper argues that in an ensemble-based adversarial attack the surrogate models themselves should be treated as tunable, and that adversarially augmenting each surrogate Vision Transformer makes the ensemble more diverse and the resulting perturbations more transferable. The proposed ViT-EnsembleAttack clones each surrogate into three stochastic variants: attention heads are randomly dropped, attention-score matrices are randomly scaled, and MLP feature maps are randomly mixed, with the randomness parameters selected by Bayesian optimization. The augmented variants are ensembled with automatic loss reweighting and a tripled step size to generate adversarial examples. On a 1000-image validation sample the method reports average attack success gains over the strongest prior baseline of about 4.6 percentage points against ViT targets and 15.3 points against CNN targets, including adversarially trained models. The reason to care is that transfer-based black-box attacks are the practical threat model for unseen classifiers, and this is a concrete route to making those attacks considerably stronger on transformer architectures.","feed_headline":"Augmented ViT ensembles push attack transfer rates past 99%","feed_subtitle":"Random head dropping, attention scaling, and MLP mixing in surrogates lift transfer success by up to 15.3 points.","key_machinery":"The load-bearing object is the augmented ViT ensemble built from three stochastic structural perturbations, each an analogue of a data augmentation: masking (multi-head dropping), scaling (attention-score scaling), and mixup (MLP feature mixing). These create different backpropagation paths for the same input, so the accumulated gradient is an average over diverse network states rather than over one fixed architecture; the paper identifies that diversity as the source of transferability. Bayesian optimization tunes the randomization parameters against the remaining surrogates, automatic reweighting ($w_i \\propto (L_{\\max}/L_i)^b$) equalizes loss magnitudes across models so no single surrogate dominates the gradient, and step-size enlargement ($\\alpha = q\\epsilon/T$) drives the attack closer to convergence within ten iterations. Formally, the update is $x_{t+1}=x_t+\\alpha\\,\\mathrm{sign}(g_{t+1})$ with momentum $g_{t+1}=\\mu g_t + \\nabla_{x_t}\\sum_i w_i L_i$, where $L_i$ sums the losses of the three augmented variants of surrogate $i$ over two stochastic inference passes.","core_discovery":"The central claim is that the transferability bottleneck for ensemble attacks on ViTs lies in the surrogate models' lack of diversity, not in the ensemble combination rule. To remove that bottleneck, each original surrogate $f_i$ is replaced by three parameterised variants: Multi-head dropping ($f^{\\mathrm{MHD}}_{\\tau_i}$) zeroes attention heads whose random threshold falls below $\\tau_i$; Attention score scaling ($f^{\\mathrm{ASS}}_{s_i,\\xi_i}$) multiplies each attention-score matrix elementwise by a random factor in $[s_i-\\xi_i, s_i+\\xi_i]$; and MLP feature mixing ($f^{\\mathrm{MFM}}_{\\rho_i}$) interpolates the MLP output with a randomly permuted copy using weight $\\rho_i$. Bayesian optimization chooses the parameters by generating MI-FGSM adversarial examples on each variant and measuring their success against the other original surrogates. At attack time all variants contribute to a momentum gradient update whose ensemble weights are recomputed from per-model losses, and the step size is enlarged to $q\\epsilon/T$ with $q=3$. The paper reports that this pipeline outperforms SVRE, AdaEA, and SMER on eight ViT targets and eight CNN targets under I-FGSM, MI-FGSM, DI-FGSM, and TI-FGSM, and that the model-augmentation module is the largest single contributor to the gain.","pith_inferences":["If the missing projection in Algorithm 2 is not a typo, the unclipped update with $\\alpha=4.8$ and $T=10$ can reach a per-pixel change of 48, three times the stated $\\epsilon=16$ budget; re-running the comparisons with explicit clipping is the minimal check on whether the 15.3-point CNN margin is budget-fair.","The paper's stated rationale predicts that the gain comes from diversity of the gradient-computation paths, not from anything attention-specific; a natural test is to apply analogous random channel dropping and feature-map mixing to CNN surrogates and see whether the transferability gain persists.","Bayesian optimization here only needs the other surrogates as validation targets, so the same machinery could be used with a single surrogate by holding out some of its own augmented variants, making the attack usable when only one surrogate is available.","The step-size enlargement result suggests a broader recipe for transfer attacks: within a fixed iteration count, larger-than-classical steps with a stabilizing mechanism can help escape adversarial overfitting, a hypothesis the paper only tests through its $q$ sweep."],"forward_implications":["With DI-FGSM integration, the method reports near-saturating transfer to ViT targets, with attack success rates at or above 99% on most of the eight tested ViTs.","Across eight CNN targets—including adversarially trained Inception ensembles and a hybrid ViT-CNN model—the method reports an average attack success rate of 88.3%, which is 15.3 percentage points above the SMER baseline.","Ablation results show that model augmentation alone raises average success from 70.5% to 93.4% on ViTs and from 48.1% to 78.8% on CNNs, making it the dominant module.","Automatic reweighting and step-size enlargement each improve over the baseline when used alone and are complementary to augmentation; combining all three modules gives the best reported results, 98.4% on ViTs and 88.3% on CNNs.","Because model augmentation is a one-time preprocessing phase, the per-image attack cost in the second phase is comparable to or lower than that of prior ensemble baselines."],"supporting_citations":[{"why":"Supplies momentum-accelerated FGSM, the gradient-update engine used both in the Bayesian-optimization objective and in the final attack loop.","marker":"[3]"},{"why":"Establishes ensembling on logits and introduces translation-invariant FGSM, one of the four attack integrations tested.","marker":"[4]"},{"why":"Defines the Vision Transformer blocks (multi-head self-attention and MLP) that the three augmentation strategies modify.","marker":"[5]"},{"why":"Provides the iterative FGSM baseline that the paper integrates and to which it compares.","marker":"[17]"},{"why":"Defines the adversarially trained ensemble targets used to test cross-architecture transfer to defended CNNs.","marker":"[35]"},{"why":"Documents the relative robustness of ViTs, motivating the paper's focus on ViT-specific surrogate augmentation.","marker":"[41]"},{"why":"Input-diversity attack with which the augmented ensemble reaches near-saturation transfer to ViT targets.","marker":"[44]"},{"why":"An adaptive ensemble-reweighting baseline; the paper contrasts its weight-focused solution with augmenting the surrogate models themselves.","marker":"[1]"},{"why":"The strongest prior ensemble baseline, which supplies the default hyperparameters and the main comparison for the 4.6% and 15.3% margins.","marker":"[32]"},{"why":"A variance-reduced ensemble baseline whose gradient-regularization approach is replaced here by diversifying the surrogate models.","marker":"[45]"}],"fun_headline_variants":["Augmented ViT surrogates lift ensemble attack transfer","ViT attack transfer boosted by augmented surrogate models","Adversarial augmentation improves ViT ensemble transferability","Three ViT augmentation strategies strengthen ensemble attacks","Model augmentation, not weight tuning, drives ViT transfer gains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the enlarged step size still respects the stated per-pixel perturbation budget, because Algorithm 2 prints the update $x_{t+1}=x_t+\\alpha\\,\\mathrm{sign}(g_{t+1})$ with $\\alpha=4.8$ over ten iterations and no explicit projection or final clipping, which would allow a raw perturbation of up to 48 against the declared $\\epsilon=16$.","fun_headline_variants_meta":{"raw":{"variants":["Augmented ViT surrogates lift ensemble attack transfer","ViT attack transfer boosted by augmented surrogate models","Adversarial augmentation improves ViT ensemble transferability","Three ViT augmentation strategies strengthen ensemble attacks","Model augmentation, not weight tuning, drives ViT transfer gains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000629,"raw_usage":{"total_tokens":2977,"prompt_tokens":1083,"completion_tokens":1894,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":699,"completion_tokens_details":{"reasoning_tokens":1818}},"tokens_in":699,"tokens_out":1894,"duration_ms":19946,"temperature":1.0,"reasoning_tokens":1818,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:22:06.911045+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released code at the paper's settings and measure the $\\ell_\\infty$ norm of every final adversarial image; if many exceed 16, re-run the Table 1 and Table 2 comparisons with per-iteration projection onto $\\{\\delta:\\|\\delta\\|_\\infty\\le 16\\}$ and check whether the reported 4.6-point and 15.3-point margins survive.","supporting_citations":[{"cited_title":"Boosting adversarial at- tacks with momentum","cited_arxiv_id":null,"evidence_quote":"Supplies momentum-accelerated FGSM, the gradient-update engine used both in the Bayesian-optimization objective and in the final attack loop."},{"cited_title":"Evading defenses to transferable adversarial examples by translation-invariant attacks","cited_arxiv_id":null,"evidence_quote":"Establishes ensembling on logits and introduces translation-invariant FGSM, one of the four attack integrations tested."},{"cited_title":"Ad- versarial examples in the physical world","cited_arxiv_id":null,"evidence_quote":"Provides the iterative FGSM baseline that the paper integrates and to which it compares."},{"cited_title":"Towards transferable adversarial attacks on vision transformers","cited_arxiv_id":null,"evidence_quote":"Documents the relative robustness of ViTs, motivating the paper's focus on ViT-specific surrogate augmentation."},{"cited_title":"Improving transferabil- ity of adversarial examples with input diversity","cited_arxiv_id":null,"evidence_quote":"Input-diversity attack with which the augmented ensemble reaches near-saturation transfer to ViT targets."},{"cited_title":"An adaptive model ensemble adversarial attack for boosting adversarial transferability","cited_arxiv_id":null,"evidence_quote":"An adaptive ensemble-reweighting baseline; the paper contrasts its weight-focused solution with augmenting the surrogate models themselves."},{"cited_title":"Ensemble diversity facilitates adversarial transferability","cited_arxiv_id":null,"evidence_quote":"The strongest prior ensemble baseline, which supplies the default hyperparameters and the main comparison for the 4.6% and 15.3% margins."},{"cited_title":"Stochastic variance reduced ensemble adver- sarial attack for boosting the adversarial transferability","cited_arxiv_id":null,"evidence_quote":"A variance-reduced ensemble baseline whose gradient-regularization approach is replaced here by diversifying the surrogate models."}],"review_version":2}