{"id":"21ab9e03-75b9-4a64-a84b-c2e0ad448a0d","arxiv_id":"2607.03839","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Mixing clean loss with FGSM-style adversarial stability inside LassoNet improves OOD accuracy and feature-support reproducibility under nearly matched sparsity, with larger medical gains only when the feature budget is not too tight.","lead":"AdLNet adds first-order input adversarial perturbations to LassoNet so sparse feature selection favors features that stay predictive under local worst-case noise. It is a practical robustness tweak for high-dimensional selection, with mixed but real gains on medical spectra and a color-shortcut benchmark.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"SERS headline gains rest on unmatched sparsity; Table 7 shows adversarial training can hurt under tight budgets, so the medical claim is not yet isolated from feature-count effects.","rationale":"The reader’s weakest assumption—that local ℓ2 adversarial sensitivity is a reliable proxy for pruning spurious rather than merely perturbation-sensitive features—is exactly the load-bearing premise. Table 7 already supplies the counter-example under tight budgets, and Table 1’s main SERS comparison is confounded by feature count. ColoredMNIST (Table 3) is cleaner (nearly matched sparsity, +4.4% OOD, +6.3% Jaccard) and the public-benchmark results under k=50 are competitive, so the paper still supports a modest positive claim for the method as a stability-aware extension of LassoNet. No internal mathematical inconsistency is evident in the first-order approximation or the NTK growth condition; those are correctly presented as local/diagnostic. Code availability further supports CONDITIONAL rather than REJECT. The concrete matched-k path on SERS is the single check that would settle whether the medical headline survives once feature count is controlled. Verdict therefore stays CONDITIONAL; no upgrade or downgrade is warranted until that isolation is shown.","tokens_in":15862,"tokens_out":640,"duration_ms":6011,"concrete_test":"Re-run the full SERS regularization path for both LassoNet and AdLNet, then report test accuracy/AUC/Sens/Spec at every matched k in {100,120,150,200,250,300} (same seeds, same α selected only on validation). If AdLNet’s advantage disappears or reverses for all k≤200, the abstract’s 5.3%/6.0% medical claim is not sparsity-isolated and should be restated as budget-dependent.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper’s strongest applied claim is the SERS lung-cancer result: AdLNet improves test accuracy by ~5.3% and AUC by ~6.0% over vanilla LassoNet (Abstract; Table 1). That comparison is not sparsity-matched: LassoNet selects 119 features while AdLNet selects 256 under the same soft constraint n_selected < 300. The controlled budget analysis (Appendix Table 7) shows the opposite pattern under high sparsity (k≈120): adversarial training collapses test accuracy (72.14 → 52.86) and AUC (0.775 → 0.482). Under moderate sparsity (k≈250) AdLNet wins while selecting slightly fewer features. Thus the medical headline is load-bearing on the premise that the mixed objective L_mix preferentially retains stable predictive features rather than simply allowing a denser support under the validation-selected checkpoint. ColoredMNIST is nearly sparsity-matched and supports a modest OOD/reproducibility claim, but the SERS numbers that drive the abstract are not isolated from feature-count confounds. The reader already flags this; it remains the single softest point for the central applied claim.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes Adversarial LassoNet (AdLNet), which augments LassoNet’s hierarchical sparsity with a mixed objective that balances clean predictive loss and first-order input-space adversarial stability (Eqs. 3–7, Algorithm 1). Under a local Lipschitz-gradient assumption, the authors derive a residual-bounded first-order approximation of local adversarial sensitivity and give an NTK-inspired spectral analysis (Lemma 2, Proposition 3) linking perturbation-driven training to increased effective rank when a spectral growth score is positive. Empirically, AdLNet is competitive with sparse baselines on six public datasets at a matched budget k=50, improves ColoredMNIST OOD accuracy and Jaccard support reproducibility under nearly matched sparsity, and reports gains on a high-dimensional SERS lung-cancer dataset versus vanilla LassoNet, with code released.","tokens_in":16228,"tokens_out":792,"duration_ms":6097,"significance":"If the stability-driven selection story holds, the work is a useful and practical bridge between hierarchical sparse feature selection and adversarial robustness, especially for high-dimensional scientific/medical settings where support reproducibility matters. Strengths include a clean residual bound (Appendix A), preservation of LassoNet’s hierarchical proximal pipeline rather than a new sparsifier, public code/dataset, and sparsity-matched ColoredMNIST evidence for modest OOD and reproducibility gains. The NTK spectral diagnostics are a thoughtful geometric interpretation even if only diagnostic. The main applied significance hinges on whether SERS gains can be isolated from feature-count effects; that isolation is currently incomplete.","major_comments":[{"comment":"Abstract and Table 1 claim a 5.3% test-accuracy and 6.0% AUC gain on SERS over vanilla LassoNet, but the comparison is not sparsity-matched (119 vs 256 selected features under n_selected<300). Appendix Table 7 shows that under a high-sparsity budget (k≈120) the adversarial model collapses (test acc 72.14→52.86, AUC 0.775→0.482), while under moderate sparsity (k≈250) it wins while selecting slightly fewer features. The medical headline is therefore not yet isolated from feature-count confounds. Please either (i) report the main SERS comparison under a fixed matched budget with the same validation protocol, or (ii) substantially qualify the abstract/Table 1 claim and lead with the controlled-budget analysis.","section":null},{"comment":"The load-bearing premise (Sections 3.1–3.3) is that local worst-case ℓ2 sensitivity (Eqs. 3–6) preferentially identifies spuriously correlated features so that hierarchical proximal updates prune shortcuts rather than useful but perturbation-sensitive signal. ColoredMNIST supports a modest OOD/reproducibility effect under matched sparsity, but the SERS high-sparsity failure in Table 7 shows the proxy can remove predictive signal when the budget is tight. The paper needs a clearer discussion of when the proxy is expected to help versus hurt, and ideally a diagnostic that separates “stable predictive support” from “smoother denser predictor.”","section":null},{"comment":"Section 4.4 and Proposition 3: the NTK analysis is local and first-order (Θ̃(ρ)=Θ+ρE+R(ρ)), while Ĝ_macro compares two fully trained checkpoints on clean inputs. The manuscript already labels this diagnostic, but the abstract and conclusion still present “NTK-inspired spectral analysis” as characterizing how training reduces gradient concentration. Please keep the claim strictly diagnostic, or provide a more direct empirical check of the local growth condition G(E;Θ) rather than only the macroscopic post-hoc score.","section":null}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is a clean, implementable extension of LassoNet: keep the hierarchical proximal path, replace the loss with a mixed clean/adversarial objective using the standard first-order ℓ2 perturbation, and report both OOD accuracy and support Jaccard. That combination is new relative to the cited pieces (LassoNet, linear Adversarial Lasso, gradient-norm penalties). Code and data are linked, which helps.\n\nWhat works: public benchmarks at matched k=50 stay competitive or better; ColoredMNIST is nearly sparsity-matched (0.969 vs 0.966) and shows a real ~4.4% OOD lift and higher Jaccard; the first-order residual bound in Appendix A is correct under the local Lipschitz assumption; the NTK effective-rank story is presented as a local diagnostic, not a proof, and the Hessian numbers line up directionally. Ablations on Mice Protein show the directional perturbation beats a plain gradient-norm penalty. This is honest engineering for people who already use hierarchical sparse nets and want a stability knob.\n\nSoft spots, in proportion: the medical headline is the softest. Table 1 compares 119 vs 256 features under the same soft n_selected < 300; Appendix Table 7 shows adversarial training collapses accuracy/AUC under a tight ~120 budget and only wins under a moderate ~250 budget. So the 5.3%/6.0% abstract claim is not isolated from feature-count effects. The load-bearing premise—that local worst-case sensitivity preferentially prunes shortcuts rather than useful but fragile signal—is only indirectly tested. α, ρ, M, λ are free and validation-tuned; no strong OOD baselines beyond vanilla LassoNet. None of that sinks the method paper, but it does mean the SERS claim needs a matched-budget rewrite before anyone cites the medical numbers.\n\nWho it is for: sparse deep feature selection and high-dimensional applied ML people who already live in the LassoNet neighborhood. It deserves a serious referee; the math is readable, the experiments are transparent enough, and the core recipe is falsifiable. I would engage, cite the ColoredMNIST/support-reproducibility angle if I am writing on stable sparse selection, and ask for sparsity-matched SERS tables.","headline":"Solid LassoNet + first-order adversarial mix with code and honest diagnostics; ColoredMNIST gains are real and sparsity-matched, but the SERS abstract numbers rest on unmatched feature counts and Table 7 shows the method can hurt under tight budgets.","tokens_in":16796,"tokens_out":568,"would_cite":true,"duration_ms":5743,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Mixing local adversarial perturbations into hierarchical sparse selection yields more reproducible features and better out-of-distribution accuracy without denser supports.","keywords":["sparse feature selection","adversarial training","LassoNet","hierarchical sparsity","out-of-distribution robustness","support reproducibility","Neural Tangent Kernel","SERS"],"falsifier":"On a controlled spurious-correlation benchmark with matched feature budgets, if AdLNet’s selected supports show no higher OOD accuracy or Jaccard reproducibility than vanilla LassoNet (or systematically drop known stable features while retaining color shortcuts), the stability-driven selection claim fails.","tokens_in":16749,"feed_emoji":"🛡️","tokens_out":601,"duration_ms":5709,"temperature":0.7,"pith_summary":"Standard sparse feature selection optimizes for clean training accuracy and can lock onto shortcut variables that fail under noise or distribution shift. This paper argues that the same hierarchical sparsity mechanism used by LassoNet can be steered toward stable features by adding a first-order worst-case input perturbation to the training objective. The mixed loss keeps the original proximal sparsity pipeline intact while the perturbation signal preferentially weakens features whose predictive contribution collapses under small local changes. On a high-dimensional medical spectroscopy task and a controlled color-spurious digit benchmark, the method keeps competitive sparse accuracy, lifts out-of-distribution accuracy, and makes the selected feature sets more consistent across random seeds. A spectral diagnostic further links the gain to a less concentrated empirical Neural Tangent Kernel and milder Hessian curvature.","feed_headline":"Adversarial sparsity picks stabler features under shift","feed_subtitle":"First-order input attacks inside LassoNet raise OOD accuracy and support agreement without denser sets","key_machinery":"The mixed objective L_mix = (1−α)L(clean) + α L(x + r*), where r* is the closed-form first-order ℓ2 adversarial direction ρ ∇_x ℓ / (‖∇_x ℓ‖ + ε), optimized by alternating perturbation construction with LassoNet’s hierarchical proximal updates; an NTK effective-rank expansion supplies a local spectral-growth condition that explains reduced gradient concentration.","core_discovery":"Integrating a tractable first-order adversarial stability term into LassoNet’s hierarchical sparsity objective produces feature supports that remain competitive on clean sparse selection while improving out-of-distribution robustness and cross-run support reproducibility under nearly matched sparsity, with corresponding gains on a high-dimensional lung-cancer SERS screening task.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Adversarial LassoNet stabilizes sparse supports under shift","First-order attacks inside LassoNet raise OOD accuracy and agreement","AdLNet keeps sparsity while lifting robustness and reproducibility","Input-space perturbations harden hierarchical feature selection","Stability term in LassoNet improves support consistency across runs"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The method assumes that features most sensitive to small worst-case input changes are the ones that should be pruned as unstable shortcuts, rather than useful but fragile signal.","fun_headline_variants_meta":{"raw":{"variants":["Adversarial LassoNet stabilizes sparse supports under shift","First-order attacks inside LassoNet raise OOD accuracy and agreement","AdLNet keeps sparsity while lifting robustness and reproducibility","Input-space perturbations harden hierarchical feature selection","Stability term in LassoNet improves support consistency across runs"]},"model":"grok-4.5","effort":"low","cost_usd":0.003058,"raw_usage":{"total_tokens":1093,"prompt_tokens":779,"num_sources_used":0,"completion_tokens":64,"cost_in_usd_ticks":30580000,"prompt_tokens_details":{"text_tokens":779,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":250,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":779,"tokens_out":64,"duration_ms":2649,"temperature":1.0,"reasoning_tokens":250,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T23:35:21.610605+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a controlled spurious-correlation benchmark with matched feature budgets, if AdLNet’s selected supports show no higher OOD accuracy or Jaccard reproducibility than vanilla LassoNet (or systematically drop known stable features while retaining color shortcuts), the stability-driven selection claim fails.","supporting_citations":[],"review_version":1}