{"id":"2d1e612a-d9c0-4912-a46a-1f45e8ca276a","arxiv_id":"2505.11134","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A new gradient projection method, DEP, reduces the largest Hessian eigenvalue during SNN training and improves robustness to heterogeneous data poisoning and adversarial attacks.","lead":"This paper shows that spiking neural networks trained with direct encoding and backpropagation through time can collapse catastrophically when a single training batch comes from a different data distribution. The authors propose DEP, a hyperparameter-free gradient projection method that removes the dominant gradient component, and report improved robustness across several datasets and attack settings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (18) bounds a Rayleigh quotient along one projected update direction; it does not establish a reduction in the Hessian spectral radius, and the bound itself relies on an unproven identification of gradient singular vectors with Hessian eigenvectors.","rationale":"The reader's weakest assumption (alignment of gradient singular vectors with Hessian eigenvectors) is real and important, but I see an even more basic gap: Eq. (18), even fully granted, bounds only the Rayleigh quotient of H along the DEP-updated gradient direction. It does not establish a reduction in the spectral radius ρ(H), which is the quantity the paper invokes to explain collapse and to justify DEP. The empirical results are not in dispute here; Table 1 and Fig. 3 show meaningful robustness gains, and the paper does reasonably well at ruling out gradient obfuscation. The problem is that the proposed mechanism is not established by the proof as written, and Table 2's lower final ρ(H) could arise from many gradient-modification schemes. This is exactly the sort of gap that warrants a conditional rather than unconditional acceptance: the method may work, but the theoretical guarantee stated in Eqs. (17)-(18) is unsupported as presented. A checkpoint-level measurement of alignment and of κDEP versus λ2 would directly test whether the proof's premise holds in practice, so I keep the verdict at CONDITIONAL.","tokens_in":15371,"tokens_out":7604,"duration_ms":76958,"concrete_test":"At a fixed CIFAR-10 VGG-11 checkpoint under the paper's training setup, compute the exact or approximate top two Hessian eigenvectors q1 and q2, the gradient's leading singular vectors u1 and v1, the alignment c=|vec(u1 v1^T)^T q1|, and the Rayleigh quotients κstd and κDEP of the raw and projected gradients against H. If c is not close to 1, or if κDEP>λ2, Eq. (18) fails on the actual model; if c≈1 and κDEP≤λ2, the mechanism is at least empirically consistent at that checkpoint.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 3.2's proof that DEP reduces the Hessian spectral radius has an invalid step. Eq. (17) defines κstd as the Rayleigh quotient of H along the vectorized raw gradient and says κstd≈λ1. Eq. (18) then shows κDEP≤λ2 for the projected gradient. Even if the algebra were correct, κDEP is a curvature along one particular update direction, not ρ(H)=max_{x≠0} x^T H x / ||x||^2. A bound on one projected direction says nothing about the maximum over all directions, so 'DEP strictly reduces the spectral radius of the Hessian' does not follow. The bound itself also depends on treating the gradient's singular vectors u_i,v_i as the Hessian's eigenvectors q_i: Eq. (17) calls u1v1^T 'the principal Hessian eigenvector' and Eq. (18) expands in the basis u_i v_i^T. For a general neural-network Hessian there is no reason these bases coincide, so even the κDEP≤λ2 claim is unproven. Table 2's lower measured ρ(H) after training is weaker support: DEP changes the trajectory, and any gradient perturbation could plausibly smooth the final landscape; it does not validate the proposed mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies a vulnerability of spiking neural networks (SNNs) trained with direct encoding and backpropagation through time (BPTT): introducing a single batch from a slightly different data distribution can cause catastrophic model collapse. The authors attribute this to an exceptionally large Hessian spectral radius produced by repeated inputs in direct encoding and multiplicative gradient accumulation in BPTT. They propose Dominant Eigencomponent Projection (DEP), a hyperparameter-free method that removes the dominant singular component of the gradient before each update, claiming this reduces the Hessian spectral radius and prevents sharp minima. The empirical evaluation is extensive: on CIFAR-10/100, TinyImageNet, and ImageNet, DEP improves robustness against FGSM and PGD attacks in both vanilla and adversarial training, reduces inference-time Hessian eigenvalues, and partially mitigates collapse under batch-level heterogeneous poisoning. The paper includes a gradient-obfuscation checklist and black-box attack results.","tokens_in":15626,"tokens_out":5817,"duration_ms":55667,"significance":"The paper identifies a practically important and previously underappreciated failure mode of SNNs trained with direct encoding and BPTT, and it proposes a simple, cheap, and hyperparameter-free mitigation with sizeable robustness gains in multiple settings. The empirical study is broad and includes white-box and black-box attacks, a Hessian analysis, and a heterogeneous-training protocol with explicit poisoning settings. If the empirical claims hold, DEP is a useful addition to the SNN robustness toolbox. However, the paper's central theoretical contribution—the root-cause analysis and the proof that DEP reduces the Hessian spectral radius—is not rigorous. The derivation in Sec. 3.2 relies on an unproven alignment between gradient singular vectors and Hessian eigenvectors, and the quantity bounded in Eq. (18) is not the spectral radius. The empirical Hessian measurements (Table 2) are consistent with the mechanism but do not establish it. The paper would be acceptable if the theory is substantially weakened or corrected and the claims are brought in line with what is actually shown.","major_comments":[{"comment":"The derivation treats the left and right singular vectors u_i, v_i of the gradient matrix as if they were eigenvectors of the Hessian H(θ). Eq. (17) asserts κ_std ≈ λ_1 based on alignment of the gradient with 'the principal Hessian eigenvector u_1 v_1^T', and Eq. (18) expands the projected gradient in the basis {u_i v_i^T} to compute the Rayleigh quotient. For a general neural-network Hessian there is no reason for the gradient's singular vectors to coincide with Hessian eigenvectors, and no proof is supplied. Without this identification, the inequality κ_DEP ≤ λ_2 is not established.","section":"§3.2, Eqs. (16)-(18)"},{"comment":"Even if the algebra of the Rayleigh quotient were correct, κ_DEP is the curvature along one particular update direction, namely v^T H v / ||v||^2. It is not the spectral radius ρ(H) = max_{x≠0} x^T H x / ||x||^2. Bounding the curvature along the projected gradient says nothing about the maximum over all directions, so the claim that 'DEP strictly reduces the spectral radius of the Hessian' does not follow. The authors should either replace 'spectral radius' with 'local sharpness along the update direction' or provide a genuine bound on ρ(H).","section":"§3.2, Eq. (18)"},{"comment":"The expansion on the right-hand side sums only over the 'diagonal' terms u_i v_i^T for i = 2,...,mn. The set {u_i v_i^T} is not a basis of the orthogonal complement of u_1 v_1^T; cross terms u_i v_j^T with i ≠ j also contribute to any vector orthogonal to u_1 v_1^T. Consequently, the denominator and numerator in the expression for κ_DEP are not the full projections of the gradient, and the inequality does not follow even under the (unproven) alignment assumption.","section":"§3.2, Eq. (18)"},{"comment":"The 'root cause' analysis relies on unstated approximations. In particular, Eq. (7) replaces the sum of squared gradient norms by T exp(2T λ̄_J), and Eq. (11) multiplies the BPTT and direct-encoding factors without derivation. These steps are heuristic; the paper does not prove that the Hessian spectral radius grows as claimed. The experimental observation of collapse in Fig. 2 is credible, but the theoretical attribution to an exceptionally large ρ(H) is not established by the present analysis.","section":"§3.1, Eqs. (6)-(11)"},{"comment":"With b=1, DEP already loses about 50 points of clean accuracy on CIFAR-10 under c/+p_0 (90.21% to 40.58%) and similar drops on other datasets. While the degradation is less catastrophic than the near-total failure of the vanilla baseline, it still constitutes a major collapse, which conflicts with the abstract's statement that DEP 'prevents' catastrophic model collapse. The claims in the Abstract and Section 1 should be softened to 'mitigates', and the residual vulnerability should be discussed.","section":"§4.4, Table 3"},{"comment":"The Hessian measurements are consistent with the proposed mechanism but do not validate it. DEP alters the training trajectory, so lower ρ(H) at inference could result from any form of gradient smoothing and does not confirm that the projection reduces the spectral radius during training. A controlled experiment (e.g., measuring ρ(H) before and after individual DEP updates) would be needed to support the causal claim.","section":"§4.3, Table 2"}],"minor_comments":[{"comment":"The Conclusion refers to the method as 'Domain Eigencomponent Projection'; the name used elsewhere is 'Dominant Eigencomponent Projection'.","section":"§5"},{"comment":"The abstract contains the typo 'dis crete' instead of 'discrete'.","section":"Abstract"},{"comment":"The notation vec(M(∇θL(θ))) is used in Eq. (17), but H(θ) is defined on R^{mn×mn}; the vectorization operator should be introduced explicitly before its first use.","section":"§3.2"},{"comment":"The eigenvalue ordering λ_1 > λ_2 ≥ ... assumes positive leading eigenvalues. Since Table 2 reports Pr(H) > 1, indicating negative eigenvalues among the top five, the authors should state whether |λ_1| or λ_1 is intended in the inequalities.","section":"§3.2, Eq. (16)"},{"comment":"The claim that DEP is 'hyperparameter-free' should clarify that the number of projected components is fixed to one; the projection rank could be viewed as a design choice. Also, the broader-impact statement that DEP introduces 'negligible computational overhead' would benefit from a runtime comparison against baselines.","section":"§1"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is substantial and the proposed method shows promising robustness gains, but the theoretical sections as written are not defensible. I recommend major revision rather than rejection because the method and experiments remain valuable if the theory is reframed as a heuristic explanation and the claims about spectral-radius reduction are corrected or removed. The authors should also reconcile the severity of residual collapse in Table 3 with the abstract's 'prevents catastrophic model collapse' wording."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's DEP method is worth paying attention to—it's a hyperparameter-free gradient projection that gives real robustness gains across four datasets—but the theoretical explanation of why it works doesn't hold up as written. You can cite the method on the strength of the experiments, not on the theory.\n\nWhat's actually new: the observation that SNNs trained with direct encoding and BPTT collapse when a single batch shifts distribution (hetero-training), and a simple fix: project out the top singular component of the gradient. The homogeneous training results in Table 1 are the core evidence. DEP beats strong baselines like HoSNN and FEEL by double digits under FGSM and PGD in both vanilla and adversarial training, and the black-box results in Fig. 3 show a clean trend. The gradient obfuscation checks in Sec. 4.5 are good practice—they actually rule out the most common way these numbers lie. The Hessian measurements in Table 2 do show that DEP-trained models have smaller spectral radius, which is consistent with the story.\n\nWhere it gets soft: the proof in Sec. 3.2. Eq. (18) bounds the curvature along the projected gradient direction, not the spectral radius of the Hessian. The leap from κ_DEP ≤ λ2 to 'DEP strictly reduces the spectral radius' doesn't follow. And the expansion in the basis u_i v_i^T assumes the gradient's singular vectors are the Hessian's eigenvectors, which is true only in special cases. The stress-test note is right about that. Also, the 'single backward pass' claim in Observation 1 is stronger than what the experiments actually show—they poison one batch per epoch for many epochs, which is a repeated disturbance, not a one-shot event. Finally, the hetero-training table (Table 3) has no baseline at all; sure, the vanilla SNN collapses, but you need to show that for contrast, otherwise the reader can't judge the magnitude of the mitigation.\n\nOverall: the empirical contribution is real and likely reproducible (simple method, standard benchmarks). The theoretical framing is heuristic and should be trimmed down to avoid overclaiming. This is a legitimate paper for the SNN robustness community, though it needs revision. A serious referee should engage with it—the method deserves scrutiny and probably a code release. I'd say accept if the authors reframe the theory as a plausible mechanism rather than a proof, and add the collapsed baseline to Table 3.","headline":"Empirically strong SNN robustness fix with a shaky theoretical story; the DEP method works across datasets but the claimed Hessian spectral radius reduction mechanism isn't proven.","tokens_in":16125,"tokens_out":2292,"would_cite":true,"duration_ms":21650,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Projecting away the largest singular component of each gradient prevents spiking neural networks from collapsing when training data distribution shifts.","keywords":["spiking neural networks","adversarial robustness","Hessian spectral radius","dominant eigencomponent projection","heterogeneous training","backpropagation through time","sharp minima","model collapse"],"falsifier":"Train an SNN with DEP while logging each batch the cosine similarity between the vectorized leading singular component $u_1 v_1^{\\top}$ and the Hessian's top eigenvector (computed via Hessian-vector products). If the alignment is weak through training, the claimed spectral-radius reduction in Eq. (18) is not the operative mechanism, and the robustness gains would have to be attributed to generic gradient smoothing.","tokens_in":15168,"feed_emoji":"⚡","tokens_out":7515,"duration_ms":62412,"temperature":0.7,"pith_summary":"This paper claims that spiking neural networks trained the standard way—direct encoding plus backpropagation through time—are one bad batch away from total collapse, because those two mechanisms inflate the largest eigenvalue of the loss's curvature matrix. It proposes Dominant Eigencomponent Projection (DEP), a hyperparameter-free step that computes the gradient's largest singular component each batch and projects the update orthogonal to it. The paper argues this provably lowers the Hessian's spectral radius, keeps the network away from sharp minima, and thereby protects against both deliberately poisoned training batches and adversarial test-time attacks. The reported experiments back this up across CIFAR-10/100, TinyImageNet, and ImageNet, with DEP beating existing SNN defenses under FGSM and PGD attacks while also preventing catastrophic collapse during heterogeneous training.","feed_headline":"Deleting a gradient's top direction stops SNN training collapse","feed_subtitle":"The new DEP step cuts Hessian sharpness each batch, shielding SNNs from poisoned data and adversarial attacks.","key_machinery":"The central object is the Dominant Eigencomponent Projection (DEP) operator $P_{u_1 v_1^{\\top}}(A) = \\langle A, u_1 v_1^{\\top} \\rangle_F \\, u_1 v_1^{\\top} / \\|u_1 v_1^{\\top}\\|_F^2$, applied to the matrixized gradient $M(\\nabla_\\theta L(\\theta))$ obtained by reshaping the $k$-dimensional gradient tensor into an $m \\times n$ matrix and taking its top singular vectors. Removing this component leaves an update whose quadratic form against the Hessian is at most $\\lambda_2 < \\lambda_1$, so the effective curvature along the chosen direction is strictly smaller than the spectral radius. This is what transfers the theoretical analysis—that BPTT's multiplicative gradient accumulation and direct encoding's repeated identical inputs inflate $\\rho(H)$ exponentially in the time steps $T$—into a training procedure that avoids sharp minima.","core_discovery":"DEP is the discovery: an orthogonal projection that removes from the gradient its dominant eigencomponent—the outer product $u_1 v_1^{\\top}$ of the leading singular vectors of the matrixized gradient—before the gradient is applied. The paper's central claim is that this single deterministic step reduces the Hessian spectral radius, because the dominant singular direction of the gradient is (the paper assumes) aligned with the top Hessian eigenvector; removing it forces the update to lie in the subspace of curvature bounded by the second eigenvalue $\\lambda_2$. Across clean, FGSM, and PGD evaluation on four datasets, DEP-trained SNNs show substantially lower $\\rho(H)$ and a smaller share of the spectral radius in the top-five eigenvalues, which the paper ties to smoother loss landscapes and better robustness. In heterogeneous training, where one batch per epoch is swapped for differently-distributed data, DEP keeps the model from collapsing even at poisoning strength $b = 5$, the point where the vanilla SNN baseline fails completely.","pith_inferences":["Beyond the paper: if the gradient's leading singular direction reliably approximates the top Hessian eigenvector in other temporal models, the same projection would offer a cheap regularizer against sharp minima in recurrent networks, transformers with long context, and online continual learning.","Beyond the paper: the clean-accuracy cost of DEP (roughly 3–5 points) suggests a robustness–accuracy tradeoff; combining DEP with knowledge distillation or with perturbations that only remove the dominant component on poisoned batches might reclaim most of that loss.","Beyond the paper: a direct test of the mechanism is to measure the cosine similarity between $\\operatorname{vec}(u_1 v_1^{\\top})$ and the top Hessian eigenvector during training; strong alignment would confirm the proof's premise, weak alignment would suggest the gains come from generic smoothing rather than spectral-radius reduction.","Beyond the paper: the batch-level poisoning scenario implies that attackers should concentrate perturbations into single batches; a practical extension is to evaluate DEP against adaptive attacks that interleave clean and poisoned samples across batches to try to circumvent the projection."],"forward_implications":["DEP reduces the Hessian spectral radius of trained SNNs, measured both at clean inference and under FGSM and PGD attacks, which the paper ties to smoother loss landscapes.","DEP improves white-box adversarial robustness over existing SNN defenses, with double-digit accuracy gains in several FGSM and PGD settings on CIFAR-100, TinyImageNet, and ImageNet.","In heterogeneous training, DEP prevents catastrophic model collapse even when several poisoned batches are injected per epoch, where vanilla SNNs drop to near-zero accuracy.","DEP passes the standard gradient-obfuscation checklist, meaning its robustness comes from genuine invariance rather than from masking gradients.","DEP is hyperparameter-free and adds only one SVD and one projection per training batch, with no extra inference cost."],"supporting_citations":[{"why":"Defines the spatio-temporal backpropagation (BPTT) training used for SNNs, whose gradient accumulation over time steps drives the large Hessian spectral radius the paper analyzes.","marker":"[35]"},{"why":"Supplies the direct encoding scheme whose repeated identical inputs reinforce the same parameter directions and further amplify Hessian eigenvalues.","marker":"[29]"},{"why":"Establishes the link between large Hessian spectral radius and sharp minima, the instability mechanism invoked to explain SNN collapse.","marker":"[3]"},{"why":"Defines FGSM, the single-step white-box attack used to evaluate DEP's adversarial robustness.","marker":"[13]"},{"why":"Defines PGD, the iterative white-box attack used to evaluate DEP's adversarial robustness.","marker":"[23]"},{"why":"Baseline defense whose robustness numbers DEP is compared against.","marker":"[8]"},{"why":"Baseline adversarial-defense method whose robustness numbers DEP is compared against.","marker":"[11]"},{"why":"Baseline defense whose robustness numbers DEP is compared against.","marker":"[37]"},{"why":"Provides the gradient-obfuscation checklist DEP must pass to show its robustness is not spurious.","marker":"[1]"},{"why":"Supplies the Hessian eigenvalue computation technique used to measure $\\rho(H)$ and the top-five eigenvalue proportion.","marker":"[39]"}],"fun_headline_variants":["Project out the dominant gradient component to stop SNN collapse","DEP: one orthogonal projection makes SNNs robust to poisoning","Cutting Hessian sharpness with a single gradient projection","Remove the gradient's top eigencomponent to keep SNNs stable","SNN training saved by projecting away the Hessian's top mode"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof that DEP lowers the Hessian's spectral radius depends on the top singular vectors of the gradient being aligned with the Hessian's dominant eigenvector, an alignment the paper asserts but never measures.","fun_headline_variants_meta":{"raw":{"variants":["Project out the dominant gradient component to stop SNN collapse","DEP: one orthogonal projection makes SNNs robust to poisoning","Cutting Hessian sharpness with a single gradient projection","Remove the gradient's top eigencomponent to keep SNNs stable","SNN training saved by projecting away the Hessian's top mode"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000905,"raw_usage":{"total_tokens":3890,"prompt_tokens":936,"completion_tokens":2954,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":2870}},"tokens_in":552,"tokens_out":2954,"duration_ms":18724,"temperature":1.0,"reasoning_tokens":2870,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:56:44.517403+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an SNN with DEP while logging each batch the cosine similarity between the vectorized leading singular component $u_1 v_1^{\\top}$ and the Hessian's top eigenvector (computed via Hessian-vector products). If the alignment is weak through training, the claimed spectral-radius reduction in Eq. (18) is not the operative mechanism, and the robustness gains would have to be attributed to generic gradient smoothing.","supporting_citations":[{"cited_title":"Spa tio-temporal backpropagation for training high-performance spiking neural networks","cited_arxiv_id":null,"evidence_quote":"Defines the spatio-temporal backpropagation (BPTT) training used for SNNs, whose gradient accumulation over time steps drives the large Hessian spectral radius the paper analyzes."},{"cited_title":"Conversion of continuous-valued deep networks to efficient event-driven networks for image classification","cited_arxiv_id":null,"evidence_quote":"Supplies the direct encoding scheme whose repeated identical inputs reinforce the same parameter directions and further amplify Hessian eigenvalues."},{"cited_title":"En hancing the robustness of spiking neural networks with stochastic gating mechanisms","cited_arxiv_id":null,"evidence_quote":"Baseline defense whose robustness numbers DEP is compared against."},{"cited_title":"Feel-snn: Robust spiking neural networks with frequency encoding and evolutionary l eak factor","cited_arxiv_id":null,"evidence_quote":"Baseline defense whose robustness numbers DEP is compared against."},{"cited_title":"Obfu scated gradients give a false sense of security: Circumventing defenses to adversarial exampl es","cited_arxiv_id":null,"evidence_quote":"Provides the gradient-obfuscation checklist DEP must pass to show its robustness is not spurious."},{"cited_title":"Pyhessian: Neural networks through the lens of the hessian","cited_arxiv_id":null,"evidence_quote":"Supplies the Hessian eigenvalue computation technique used to measure $\\rho(H)$ and the top-five eigenvalue proportion."}],"review_version":1}