{"id":"29596bb2-b024-4eb0-9b39-865199909629","arxiv_id":"2508.16937","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"NAT trains per-neuron adversarial generators that each disrupt one mid-layer neuron, improving cross-model and cross-domain attack transferability over embedding-level baselines.","lead":"This paper trains separate generators to attack individual neurons in a vision model's mid-layer, rather than attacking the whole layer at once. These neuron-specific attacks transfer to other models more often than previous methods, improving fooling rates on 41 ImageNet models and in cross-domain tests.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Because NAT removes ReflectionPad2d from the generator while LTP/BIA baselines keep it, the reported 14% cross-model gain may partly stem from this architectural change rather than neuron-specific targeting.","rationale":"The reader's weakest assumption was the stability of the neuron ranking after lightweight training and the held-out model being inside the evaluation set. These are reasonable concerns, but the paper's own data partially mitigates them: the top-1 generator G250 selected after lightweight training is not the best after full training (G391 is about 4 points better), so ranking instability makes the reported single-query number conservative rather than inflated. Excluding the held-out DenseNet121 from the 41-model average lowers the headline by less than one point, and the ablation with ResNet152 and VGG16 as held-out models produces nearly identical results. The more load-bearing issue is that the generator architecture is changed for NAT while the strongest baselines (LTP, BIA) are evaluated with their original ReflectionPad-containing generators. The paper's Section 6 explicitly describes removing ReflectionPad2d to ensure reproducibility, but it does not retrain the baselines with this same modification. Since padding type can change boundary artifacts and transferability, the observed 14% gain cannot be cleanly attributed to the neuron-specific objective. This is a concrete, testable confound that directly affects the central claim. A secondary issue is the cross-domain claim: Table 5 reports adversarial accuracy (lower is better), yet the abstract describes a '4% improvement' in fooling rate, a mismatch that appears to be a reporting error rather than a core validity threat. Overall, the conditional verdict remains appropriate, with the added requirement to control the generator architecture in the baseline comparison.","tokens_in":27967,"tokens_out":11685,"duration_ms":110634,"concrete_test":"Retrain LTP and BIA with the exact same generator architecture as NAT (ReflectionPad2d removed, padding set as described in Section 6), using the same training data, hyperparameters, and perturbation budget, and re-run the Table 2-4 evaluation. If the padding-free LTP/BIA match NAT's 60.6% average fooling rate, the neuron-specific claim is unsupported; if they remain near 44.6%, the architectural confound is minor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central comparison in Section 4.1 is not architecture-controlled. In Section 6 ('Generator Modifications'), the authors state that they removed the non-deterministic ReflectionPad2d layers from the generator and replaced them with ordinary padding. However, the LTP and BIA baselines are used as pretrained generators from the original papers, which retain ReflectionPad2d. The paper does not report retraining these baselines with the padding-free architecture. Padding type is known to affect boundary artifacts and can materially influence the transferability of generated perturbations. Consequently, the 14-point single-query improvement over LTP in Figure 4 and Tables 2-4 could be partly or largely due to this architectural change rather than to the neuron-specific loss in Eq. (1). The cross-domain comparisons in Table 5 have the same confound. This is the weakest link in attributing the headline number to neuron-level targeting; the reader's ranking-stability concern is secondary because Tables 9/10 show that the selected top-1 generator is not even the best after full training, meaning the reported single-query result is conservative rather than inflated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NAT, a generative transferable-attack method that trains individual generators to maximize L2 separation of a single neuron (channel) in a mid-layer embedding of a VGG16 source model, rather than maximizing separation of the full embedding as in LTP/BIA. Using a low-fidelity warm-up on 3.12% of ImageNet, the authors select 40 of 512 neuron-specific generators and fully train them; at inference they query the target with one or several of these generators. The central empirical claim is that a single NAT generator achieves about 60.6% average fooling rate over 41 ImageNet models versus 44.6% for LTP, and that querying up to 10 generators improves transferability further. The paper also reports cross-domain results on three fine-grained datasets.","tokens_in":28143,"tokens_out":4026,"duration_ms":43705,"significance":"If the reported gains hold under a controlled comparison, the paper would be a meaningful step in understanding why embedding-level generative attacks transfer: shifting from whole-embedding objectives to per-neuron experts is a clean, simple idea, and the experiments are substantially broader than prior work (41 ImageNet models instead of about five). The full per-generator/per-model tables in the appendix, the public code release, and the explicit discussion of generator-level failure modes are strengths. However, the headline improvement is currently difficult to attribute to neuron-specific targeting because the NAT generator architecture differs from the baselines in a way that is known to affect perturbation patterns, and the neuron-selection protocol uses models that are also in the evaluation set.","major_comments":[{"comment":"The comparison is not architecture-controlled. The paper states that it removed the non-deterministic ReflectionPad2d operations from the generator and replaced them with ordinary padding, while the LTP and BIA baselines are the original pretrained generators that retain ReflectionPad2d. Because padding type changes boundary artifacts and can materially affect the transferability of generated perturbations, the reported 14-point single-query improvement over LTP may be partly or largely due to this architectural change rather than to the neuron-specific loss in Eq. (1). Please retrain LTP and BIA with the identical padding-free generator architecture, or provide an ablation of NAT trained with the original ReflectionPad2d generator, and report the controlled comparison.","section":"Section 6, 'Generator Modifications'; Section 3.3; Figure 4; Tables 2-4"},{"comment":"The neuron-selection protocol is not independent of the evaluation set. In the main experiments the top-k neurons are selected using DenseNet121 as the held-out model, and DenseNet121 is also one of the 41 target models in Table 2; the ablation uses ResNet152 and VGG16, which also appear in the target list. This means the rows for these models are not held out. Although Table 6 suggests robustness across selection models, the reported averages should be recomputed excluding the selection model from the target set, or the authors should use a held-out model that is not among the 41 evaluation models.","section":"Section 4.1, Section 4.4, Tables 2-6"},{"comment":"The cross-domain claim in the abstract and Section 4.2 should be stated in a way that matches Table 5. Table 5 reports adversarial accuracy, where lower values indicate a stronger attack, and the difference between NAT (40.94%) and FACL (44.05%) is about 3.1 percentage points, not 4%; the text says 'additional 3.5% points' and the abstract says 'over 4% improvement'. Please reconcile the numbers and clearly state the metric orientation in the text.","section":"Section 4.2, Table 5"}],"minor_comments":[{"comment":"The abstract and text say 'cross-modal' where the intended term is likely 'cross-model'; please correct for consistency.","section":"Abstract and Section 4.1"},{"comment":"The text says 'around 424/512 (83%) neurons are largely unaffected with the LTA'; 'LTA' appears to be a typo for 'LTP'.","section":"Section 3.1, Figure 2"},{"comment":"There are several typos that should be cleaned up, including 'phenemenon', 'synthethesised', 'prevous', and 'stratefies'.","section":"Section 3.1"},{"comment":"The term 'ReflectionPadding layers' is used in Section 3.3 while the appendix uses 'ReflectionPad2d'; please use one consistent name.","section":"Section 6, Generator Modifications"},{"comment":"In Table 5, please add a note in the caption that lower adversarial accuracy means a stronger attack, since the values in the text otherwise appear counterintuitive.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The central idea is sound and the evaluation scale is a strong positive, but the architecture confound in the main comparison must be resolved before the headline claim can be trusted. The held-out-model leakage is more easily fixed by recomputing averages without the selection model; I do not see it as fatal, but it should be addressed explicitly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read NAT. The genuinely new thing is the per-neuron generator setup: instead of one generator maximizing embedding separation, they train a generator per channel, each maximizing L2 separation of that single neuron, and then combine the top-k ranked generators as queries. That is a real departure from LTP/BIA/GAP/CDA, and the intuition — layer-level attacks concentrate on a few concept-specific neurons — is supported by their histogram and visualizations. The evaluation is also unusually broad: 41 ImageNet models plus fine-grained cross-domain, with per-generator tables in the supplement. I believe the central phenomenon is real.\n\nThe soft spot is load-bearing, though. In Section 6 they say they removed ReflectionPad2d from the generator to make training deterministic, replacing it with ordinary padding, but the LTP/BIA baselines are used as pretrained generators that still contain ReflectionPad2d. That means the headline 14-point single-query gain over LTP is not architecture-controlled. Padding type changes boundary artifacts and is known to affect transferability, so part of the gain may come from the architectural change, not from per-neuron targeting. They never retrain the baselines with the same padding-free generator. This is the main thing I would want fixed before trusting the quantitative claim.\n\nThe selection-loop issue the reader flagged is real but less damaging. DenseNet121 is both held-out selector and in the 41-model eval set, yet the held-out ablation in Table 6 shows stable cross-model results across DenseNet121, ResNet152, and VGG16, and Tables 9/10 show the top-ranked after lightweight training is not even the best after full training (G391 beats G250), so the reported single-query number is conservative rather than inflated. The cross-domain numbers are harder to read because lower values are better in Table 5, and the abstract's \"4% improvement\" does not obviously match the table. Minor reporting issue.\n\nVerdict: conditional. The core idea deserves a serious referee. The paper ships code, neuron positions, per-generator results, and an honest limitations section, which all help. If the authors add a padding-controlled comparison — retrain LTP/BIA with the same padding-free generator, or re-run NAT with ReflectionPad2d — the quantitative claim can be assessed cleanly. I would send it to review, and I would make the confound the first-round request.","headline":"Per-neuron generators are a genuinely new attack idea with a broad evaluation, but the headline gain over LTP is confounded by an unreported padding change in the generator.","tokens_in":28690,"tokens_out":2492,"would_cite":true,"duration_ms":25864,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Attacking individual neurons instead of whole layers makes adversarial images transfer across many more models.","keywords":["adversarial transferability","generative adversarial attacks","neuron-level attacks","feature-based attacks","black-box attacks","query-efficient attacks","ImageNet","vision transformers"],"falsifier":"Take a held-out model outside the 41 target architectures, retrain the 512 lightweight generators on a different 3.12% data subset, rank them, fully train the top-1 and a bottom-ranked generator, and measure average fooling rates on the remaining targets; if the top-1 rate falls to roughly LTP's 44.6% or the ranking flips under full training, the neuron-ranking premise is false.","tokens_in":27728,"feed_emoji":"🎯","tokens_out":6754,"duration_ms":58587,"temperature":0.7,"pith_summary":"Existing generative adversarial attacks train a single generator to pull whole mid-layer embeddings apart, and this paper shows that in practice such attacks concentrate their damage on a small set of neurons encoding one concept, leaving most of the layer untouched. NAT instead trains one generator per neuron, with the sole objective of maximizing the change in that neuron's activation between clean and adversarial images. The paper reports that a single such generator fools an average of 60.6% of 41 ImageNet models, compared with 44.6% for the strongest baseline LTP, and that querying the top-40 generators in ranking order pushes fooling rates above 90% on convolutional models within 10 queries. The larger claim is that individual neurons, not layer-level representations, are the common currency that transfers across architectures.","feed_headline":"Attacking single neurons lifts adversarial transfer to 60.6%","feed_subtitle":"Training one generator per neuron beats whole-layer attacks on 41 ImageNet models and needs fewer than 10 queries.","key_machinery":"The mechanism is a family of neuron-specific generators. A 'neuron' here means one channel of a CNN feature map: layer 18 of VGG16 has 512 such channels. Each generator G_j is trained with the loss $L_{adv}(G_j) = ||f^j_l(x) - f^j_l(x_a)||_2^2$, so it attacks exactly one concept rather than the whole embedding. The critical second piece is low-fidelity selection: all 512 generators are trained briefly on 3.12% of ImageNet, ranked by fooling rate on a held-out model, and only the top 40 are fully trained; at inference the generators are queried in that rank order, and an image counts as fooled if any one of the queries succeeds. The loss gives each generator a single focused objective, and the ranking procedure converts that diversity into an ordered ensemble that works within a tight query budget.","core_discovery":"The central discovery is that transferability is better bought at the level of a single channel of a mid-layer feature map than at the level of the whole embedding. For each neuron position j in layer 18 of VGG16, NAT trains a generator G_j that maximizes $||f^j_l(x) - f^j_l(x_a)||_2^2$, where $f^j_l$ is the activation map of that neuron. After a cheap training pass on 3.12% of ImageNet to rank all 512 generators by transferability to a held-out model, the top 40 are fully trained; in the single-query setting the best-ranked generator reaches 60.6% average fooling rate across 41 architectures, over 14 percentage points above the LTP baseline's 44.6%, and with 10 or 40 queries the complementary generators reach 93.9% and 96.2% on the 16 convolutional models. In cross-domain settings the single-query attack lowers average adversarial accuracy from around 50% for prior feature-based baselines to 40.94% across nine fine-grained models. The paper argues these gains come from each generator specializing in one distinct concept, producing diverse perturbations that cover neurons other attacks leave untouched.","pith_inferences":["A stronger selection rule may be available: the paper notes the best fully trained generator G391 outperforms the top-ranked G250 by over 4%, implying the cheap 3.12% ranking leaves recoverable gains; a two-stage or diversity-aware selection could push single-query rates higher.","The ranking premise should be tested with held-out models fully disjoint from the target set, since DenseNet121, ResNet152, and VGG16 all appear among the 41 evaluated models; if the ranking advantage shrinks on a disjoint held-out model, part of the reported gain is selection leakage.","Cross-domain transfer might be improved by choosing the ranking model from the target domain's backbone family, since the paper's own ablation shows ResNet152 as held-out gives 9.5% adversarial accuracy versus 13.6% for DenseNet121 on the nine fine-grained models.","The same neuron-ranking map could serve as an interpretability diagnostic: the ordering of which neurons transfer best across 41 models is itself a measurement of how universally concepts are represented in vision networks."],"forward_implications":["Black-box attacks on unseen architectures improve with no extra information: a single NAT generator already outperforms the strongest prior baseline on 41 ImageNet models, and the gap widens on transformers and hybrid models when a few queries are allowed.","The top-40 generators behave as a small ordered ensemble, so an attacker can trade queries for fooling rate: 10 queries reach 93.9% on convolutional, 44.9% on transformer, and 74.1% on hybrid models.","Because each generator targets a distinct concept, the attack portfolio is complementary; the best generator for a target varies by architecture, so no single pattern dominates.","If neurons are genuinely shared across models, then interpretability tools that name neuron concepts could directly predict which generators to train, linking attack transferability to model alignment."],"supporting_citations":[{"why":"Provides the LTP baseline, the embedding-separation loss, the attacked layer l=18, and the 5K ImageNet evaluation subset that NAT builds on and compares against.","marker":"[60]"},{"why":"Provides the BIA baseline, the cross-domain fine-grained evaluation protocol, and the nine target models used there.","marker":"[90]"},{"why":"Supplies the interpretability result that individual neurons encode distinct concepts, motivating neuron-level attacks.","marker":"[52]"},{"why":"Corroborates that neurons can be described as interpretable concepts, used to justify targeting one neuron per generator.","marker":"[5]"},{"why":"The GAP baseline whose cross-entropy generator objective NAT contrasts with neuron-specific L2 separation.","marker":"[54]"},{"why":"The CDA baseline for relativistic cross-entropy attacks and the prior cross-domain setting NAT extends.","marker":"[51]"},{"why":"DenseNet121 is the held-out model used to rank the 512 lightweight generators and select the top-40 neurons.","marker":"[23]"},{"why":"Supports the low-fidelity strategy of ranking candidates after training on a 3.12% subset before committing to full training.","marker":"[33]"},{"why":"The activation maximization algorithm used to synthesize images of the most disrupted neurons, grounding the claim that layer-level attacks concentrate on a single concept.","marker":"[61]"}],"fun_headline_variants":["Neuron-level attacks boost adversarial transfer by 14%","Single neuron attack tops adversarial transfer at 60.6%","Targeting neurons, not layers, for stronger transferable attacks","Per-neuron generators beat layer attacks on 41 models","Attack one neuron: 60.6% fooling without queries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the ordering of the 512 neuron-specific generators by transferability, measured after training on only 3.12% of ImageNet and judged on a single held-out model such as DenseNet121, remains the right ordering after full training and on the 41 target models, even though those held-out models are themselves part of the target set.","fun_headline_variants_meta":{"raw":{"variants":["Neuron-level attacks boost adversarial transfer by 14%","Single neuron attack tops adversarial transfer at 60.6%","Targeting neurons, not layers, for stronger transferable attacks","Per-neuron generators beat layer attacks on 41 models","Attack one neuron: 60.6% fooling without queries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1643,"prompt_tokens":1005,"completion_tokens":638,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":553}},"tokens_in":621,"tokens_out":638,"duration_ms":5709,"temperature":1.0,"reasoning_tokens":553,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:10:04.859136+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out model outside the 41 target architectures, retrain the 512 lightweight generators on a different 3.12% data subset, rank them, fully train the top-1 and a bottom-ranked generator, and measure average fooling rates on the remaining targets; if the top-1 rate falls to roughly LTP's 44.6% or the ranking flips under full training, the neuron-ranking premise is false.","supporting_citations":[{"cited_title":"Learning transferable adver- sarial perturbations","cited_arxiv_id":null,"evidence_quote":"Provides the LTP baseline, the embedding-separation loss, the attacked layer l=18, and the 5K ImageNet evaluation subset that NAT builds on and compares against."},{"cited_title":"Clip-dissect: Automatic description of neuron representations in deep vision networks","cited_arxiv_id":null,"evidence_quote":"Supplies the interpretability result that individual neurons encode distinct concepts, motivating neuron-level attacks."},{"cited_title":"Generative adversarial perturbations","cited_arxiv_id":null,"evidence_quote":"The GAP baseline whose cross-entropy generator objective NAT contrasts with neuron-specific L2 separation."},{"cited_title":"Cross-domain transferability of adver- sarial perturbations","cited_arxiv_id":null,"evidence_quote":"The CDA baseline for relativistic cross-entropy attacks and the prior cross-domain setting NAT extends."},{"cited_title":"Densely connected convo- lutional networks","cited_arxiv_id":null,"evidence_quote":"DenseNet121 is the held-out model used to rank the 512 lightweight generators and select the top-40 neurons."},{"cited_title":"Hyperband: A novel bandit-based approach to hyperparameter op- timization","cited_arxiv_id":null,"evidence_quote":"Supports the low-fidelity strategy of ranking candidates after training on a 3.12% subset before committing to full training."}],"review_version":1}