{"id":"2938fd8b-b6b2-403d-aed6-27aabcdab4fa","arxiv_id":"2505.11455","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Skip recurrent connections with per-layer learned spans mitigate gradient vanishing in recurrent spiking neural networks and improve long-sequence classification accuracy.","lead":"The paper proposes replacing the standard one-step recurrent connection in spiking neural networks with a skip connection that jumps several time steps, plus an adaptive version that learns the jump length per layer. On four speech and sequence benchmarks it reports small but consistent accuracy gains over prior spiking models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8)'s temporal-gradient bound fails for matrix-valued recurrent weights: a two-neuron counterexample with ||W2||=Vth has per-step Jacobian norm sqrt(2)>1, so the vanishing-gradient motivation for SRC is not established.","rationale":"The reader's weakest assumption was empirical: reported gains might reflect hyperparameter tuning or run-to-run noise rather than the proposed mechanism. That concern is valid and should also be addressed by repeated-seed experiments. However, the most load-bearing concern is more fundamental: the analytical foundation of the central claim, Eq. (8), is mathematically incorrect in the stated vector/matrix setting. A concrete two-neuron counterexample satisfies all stated assumptions (alpha=1, Vth=1, ||W2||=Vth, surrogate derivatives in [0,1/Vth]) yet produces a per-step temporal Jacobian with spectral norm sqrt(2) > 1. Therefore the paper's assertion that temporal gradients necessarily vanish unless neurons fire at threshold is not established, and the proposed SRC/ASRC mechanism lacks its stated theoretical justification. This does not by itself prove the empirical results are wrong, and the paper could be repaired by correcting the analysis or reframing the contribution as purely empirical, but the current text overclaims. Given both the theoretical error and the absence of statistical evidence for the empirical comparisons, the central claim should be considered unverified pending revision rather than conditionally accepted on its current reasoning.","tokens_in":15432,"tokens_out":15933,"duration_ms":162694,"concrete_test":"Numerically verify Eq. (8) on the two-neuron counterexample: set alpha=1, Vth=1, W2 = [[0,1],[1,0]], surrogate derivative vector H = [1,0], and compute the spectral norm of J = alpha*I + (W2 - alpha*Vth*I)*diag(H). The resulting matrix is [[0,0],[1,1]] with singular values sqrt(2) and 0. If ||J|| > 1, the bound in Eq. (8) is false. The authors should then either re-derive the temporal Jacobian bound for vector-valued membrane potentials without assuming commutativity, or explicitly state the additional assumptions (for example, scalar recurrent weights or identical neuron states) under which their scalar bound holds.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The paper's theoretical claim that vanilla LIF-based RSNNs suffer unavoidable temporal gradient vanishing is based on Eq. (8), which bounds the temporal Jacobian product by max(alpha^k, |W2/Vth|^k) <= 1 whenever |W2| <= Vth. This derivation treats each factor alpha + (W2 - alpha*Vth)*H(U) as a scalar or as commuting operators, but W2 is a matrix and H(U) is a diagonal matrix of surrogate derivatives. The spectral norm of a sum of non-commuting matrices is not bounded in this way. Concretely, set alpha=1, Vth=1, W2 = [[0,1],[1,0]] (orthogonal, so ||W2|| = Vth), and surrogate derivatives H = [1,0] (neuron 1 at threshold, neuron 2 inactive). The per-step Jacobian from Eq. (5) is [[0,0],[1,1]], whose spectral norm is sqrt(2) > 1. Thus temporal gradients can grow across a single step even when |W2| <= Vth, and the conclusion that gradients must vanish unless every neuron fires at threshold does not follow. Because this analysis is the paper's first contribution and the stated motivation for SRC, the mechanism claim is not supported as written, even though the empirical improvements may be real.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Skip Recurrent Connection (SRC) for LIF-based recurrent spiking neural networks (RSNNs), replacing the vanilla one-step recurrence with a recurrence that spans \\lambda time steps, and an adaptive variant (ASRC) that learns the skip span per layer using a temperature-scaled softmax kernel. The theoretical motivation is a gradient analysis of vanilla LIF-based RSNNs (Eqs. 5-8) claiming that temporal gradients vanish when the recurrent weight satisfies |W2| <= Vth, and that this can only be avoided if every neuron fires at threshold at every step. Empirically, the authors report that SRC-SNN improves accuracy over vanilla RSNNs and prior SNN baselines on S-MNIST, PS-MNIST, SSC, GSC, and SCIFAR, and that ASRC-SNN further improves on SRC-SNN and shows robustness to the choice of the maximum skip span T_lambda.","tokens_in":15686,"tokens_out":3946,"duration_ms":38288,"significance":"If the empirical findings hold, the paper introduces a simple, hardware-friendly architectural modification that improves long-sequence accuracy in RSNNs, and the adaptive softmax-based selection of discrete temporal offsets is an interesting mechanism with potential extensions to spatial offsets. The authors provide code, and the experimental scope spans several benchmarks and ablations, including replacement of the LIF neuron with other neuron models and sparsity conditions. However, the theoretical gradient bound in Eqs. (5)-(8) is informal and, as shown by a matrix counterexample, not correct as stated; this weakens the central motivation for SRC. In addition, the empirical support for the superiority of ASRC over SRC is limited by the absence of multiple seeds or error bars, given that the reported differences are only 0.1-0.3 percentage points. The paper's significance therefore rests on the empirical claims, which need stronger statistical and baseline support.","major_comments":[{"comment":"The bound on the temporal Jacobian product is derived by treating the factor (alpha + (W2 - alpha Vth) H(U)) as a scalar or as a product of commuting terms, but W2 is a matrix and H(U) is a diagonal matrix of surrogate derivatives. The spectral norm of a product of non-commuting matrices is not bounded in the manner claimed. A concrete counterexample is alpha=1, Vth=1, W2 = [[0,1],[1,0]] (so ||W2|| = Vth), and H(U) = diag(1,0). The per-step Jacobian in Eq. (5) becomes [[0,0],[1,1]], whose spectral norm is sqrt(2) > 1. Thus temporal gradients can grow across a single step even when ||W2|| <= Vth, and the conclusion that gradients necessarily vanish unless every neuron fires at threshold does not follow. Since this analysis is the stated motivation for the SRC architecture, it must be corrected (for example, by using appropriate spectral-norm conditions or by weakening the claim) before the theoretical contribution can be accepted.","section":"Section 4.2, Table 1 and Figures 2-4"},{"comment":"No standard deviations or multiple seeds are reported for any result. The claimed advantage of ASRC over SRC is only 0.19 points on S-MNIST (99.57 vs 99.38), 0.10 points on SSC (81.93 vs 81.83), and 0.11 points on GSC (96.29 vs 96.18). These differences are likely within the run-to-run variance of SNN training with surrogate gradients. Without repeated trials and a paired significance test, the statement in the abstract and Section 4.3.2 that ASRC consistently outperforms SRC is not supported. Please report mean and standard deviation over at least five seeds for all main results and key ablations.","section":"Section 4.1, Table 1"},{"comment":"The baseline numbers in Table 1 are copied from prior publications with different architectures, parameter counts, data preprocessing, and training setups (e.g., TC-LIF [21], PMSN [27], DCLS-Delays [29]). The only controlled vanilla RSNN baseline appears in Figure 2 for PS-MNIST and SSC, but it is not included in Table 1 for all datasets, and its hyperparameters are not specified. Without a same-protocol vanilla RSNN trained under identical settings (same optimizer, epochs, hidden sizes, and preprocessing as SRC-SNN/ASRC-SNN), the claim that SRC significantly outperforms vanilla RSNN is not fully established. Please add such baselines for all datasets, or clearly report the existing RSNN numbers in Table 1.","section":"Section 4.1"}],"minor_comments":[{"comment":"There are several typos and inconsistent abbreviations: 'Resluts' in Section 4.2, 'hyparameter' in Section 4.3.1, 'diffirent' in the caption of Figure 4, and inconsistent spelling of 'SCIFAR' / 'SCFAR' in Appendix A.1. These should be corrected.","section":"Section 3.4.1"},{"comment":"The text says that during the testing phase Eq. (11) is used to compute p_l, but Eq. (11) is a limit expression, not an algorithm. The implementation should be described precisely, including how ties in the argmax are resolved when the temperature goes to zero.","section":"Section 5"},{"comment":"The related work and discussion should more explicitly position the contribution with respect to [40], which already analyzed skip recurrent connections, and to [47] on dilated convolutions with learnable spacings, since ASRC learns a discrete position in a similar spirit. Currently this is only mentioned as future work, which understates the prior art and should be clarified in Related Work.","section":"Section 4.3.2"},{"comment":"The 'robustness' claim for ASRC is based on insensitivity to T_lambda in Figures 2c and 2d, but the fluctuations across different T_lambda are still several tenths of a percent and no error bars are shown. Please clarify what 'robustness' means and support it with quantitative variability measures.","section":"Appendix A.2"},{"comment":"The description of the learning rate schedule for the softmax kernel would benefit from clarification: the text says the learning rate is set to 100x the global learning rate, while Table 2 gives specific values; please reconcile these and describe the schedule precisely.","section":"Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"The central theoretical claim in Section 3.2 is a load-bearing error: the matrix counterexample invalidates Eq. (8) as stated, and this is the paper's primary motivation for SRC. However, the error is fixable by either correcting the bound (e.g., using spectral radius or a proper norm condition) or by reframing the contribution as an empirical architectural improvement. The empirical claims are plausible but currently under-supported due to the lack of repeated runs, uncontrolled baselines, and the very small ASRC-vs-SRC margins. In my view the paper is suitable for major revision rather than rejection, provided the authors substantially strengthen both the theoretical and empirical support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The ASRC-SNN paper gives a simple, reproducible trick—skip recurrent connections with learned spans—and it works on the benchmarks reported. Code is public, the gains on GSC and SSC are substantial, and the pattern holds across four neuron types. That is the part I would take. The ablation generalizing SRC/ASRC beyond LIF (PLIF, GLIF, CLIF, TC-LIF) is a genuinely nice demonstration that the trick is architectural rather than tied to one neuron model.\n\nWhat is actually new is the combination: applying the λ-step recurrence from RNN literature [40] to LIF-based RSNNs, and using an annealed softmax to pick the skip span per layer. The components are known—SRC is in [40], annealed selection is in DCLS-delays—but the packaging is a legitimate engineering contribution. The citation pattern is fair; the authors do not hide those antecedents.\n\nThe soft spot is the theoretical motivation. Eq. (8) is not a valid bound when W2 is a matrix. The stress-test counterexample is correct: with α=1, Vth=1, W2=[[0,1],[1,0]], and only one neuron at threshold, the single-step Jacobian has norm √2 > 1. So the claim that temporal gradients always vanish unless every neuron fires is not established. This matters because the paper frames the gradient analysis as its first contribution. The empirical results may still be real, but the proof is not there. The paper would be stronger if the bound were fixed (e.g., spectral analysis of the full Jacobian product) or if the contribution were framed as empirical rather than theoretical.\n\nTwo smaller issues. No error bars or repeated seeds appear anywhere, and the baselines are mostly imported from other papers with different architectures and preprocessing. The ASRC gain over SRC is often 0.1–0.3 points, which is close to run-to-run noise at these near-saturation levels. Also, the appendix variant that learns a per-time-step span reaches 96.51 on PS-MNIST while the main ASRC gets 95.40; the text says that variant 'does not outperform' the original, which is hard to square with the table.\n\nWho is this for: anyone working on recurrent SNNs or neuromorphic sequence models. It deserves a serious referee, not a desk reject. I would set expectations for major revision: add repeated-seed statistics, tighten or drop the gradient claim, and rerun at least a few baselines under identical settings.","headline":"The empirical skip-connection trick is reproducible and works, but the paper's gradient-vanishing proof does not survive matrix-valued weights.","tokens_in":16303,"tokens_out":4179,"would_cite":true,"duration_ms":38910,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that LIF-based recurrent spiking networks suffer a temporal vanishing-gradient problem, and that skip recurrent connections with learned spans fix it.","keywords":["recurrent spiking neural networks","temporal gradient vanishing","skip recurrent connection","adaptive skip span","surrogate gradient","long-term temporal modeling","LIF neuron","softmax annealing"],"falsifier":"A controlled re-run with identical hidden sizes, optimizers, epochs, and per-dataset hyperparameter search across many seeds would falsify the empirical claim if SRC/ASRC do not beat the vanilla RSNN outside seed noise; separately, measuring temporal gradient norms during training would falsify the mechanism claim if SRC/ASRC do not show larger temporal gradients than vanilla RSNN.","tokens_in":15167,"feed_emoji":"🧠","tokens_out":10395,"duration_ms":89855,"temperature":0.7,"pith_summary":"This paper tries to establish why recurrent spiking neural networks (RSNNs) struggle with long sequences and what to do about it. Treating the LIF neuron and the recurrent connection as one dynamical system, it derives a bound: as long as the recurrent weight obeys $|W^l_2| \\le V_{\\mathrm{th}}$, the product of temporal Jacobians contracts, so temporal gradients vanish unless the neuron fires at every step. The proposed fix is the skip recurrent connection (SRC), which reaches back $\\lambda$ time steps instead of one, and the adaptive version (ASRC), which lets each layer learn its skip span with an annealed softmax. The reason to care is practical: this architectural change improves accuracy on sequential MNIST, permuted sequential MNIST, and speech-command benchmarks, and it works across several spiking neuron models.","feed_headline":"Skip connections fix the vanishing-memory problem in spiking RNNs","feed_subtitle":"Each layer learns its own skip lag, improving long-sequence spiking accuracy.","key_machinery":"The load-bearing object is the temporal Jacobian chain in Eqs. (7)--(8). Each adjacent-step factor is $\\alpha + (W^l_2 - \\alpha V_{\\mathrm{th}})H(U^l[t])$, and the bound $H(U) \\le 1/V_{\\mathrm{th}}$ makes each factor have modulus at most $1$ when $|W^l_2| \\le V_{\\mathrm{th}}$, so the $k$-step product contracts toward zero. The skip recurrent connection replaces $W^l_2 S^l[t-1]$ with $W^l_2 S^l[t-\\lambda]$, giving the gradient a direct path that does not multiply through the intermediate steps; the ASRC mechanism then uses a temperature-scaled softmax kernel over $T_\\lambda$ candidate lags, with temperature decaying exponentially, so each layer converges to a single chosen lag at test time. This chain-bound-and-shortcut pair is what carries the argument from gradient analysis to architecture.","core_discovery":"On the paper's own terms, the central discovery is that in a vanilla LIF-based RSNN the temporal Jacobian $\\partial U^l[t+k]/\\partial U^l[t]$ factorizes into a product of per-step terms $\\alpha + (W^l_2 - \\alpha V_{\\mathrm{th}})H(U^l[t])$, and because the surrogate spike derivative $H$ is bounded by $1/V_{\\mathrm{th}}$, the whole product obeys $|\\partial U^l[t+k]/\\partial U^l[t]| \\le \\max(\\alpha^k, |W^l_2/V_{\\mathrm{th}}|^k) \\le 1$ whenever $|W^l_2| \\le V_{\\mathrm{th}}$. Equality, which would keep gradients alive, forces the neuron to fire at every time step and destroys temporal discriminability. The paper therefore claims RSNNs are inherently prone to temporal gradient vanishing, and inserting a skip recurrent connection $S^l[t-\\lambda]$ in place of $S^l[t-1]$ gives gradients a shortcut around the contracting chain; the ASRC variant then learns which lag each layer should use by annealing a temperature-scaled softmax over $T_\\lambda$ candidate lags until it behaves as a hardmax. The empirical claim is that SRC-SNN beats vanilla RSNN by a large margin on all four benchmark datasets and that ASRC-SNN goes further, with per-layer lags that differ across layers and datasets.","pith_inferences":["Beyond the paper, the same annealed-softmax selection could learn positions in space as well as time, for instance the nonzero offsets in dilated convolution kernels, turning a manually designed dilation pattern into a learned one.","If the gradient mechanism is the true driver, SRC/ASRC networks should exhibit measurably larger temporal gradient norms than vanilla RSNNs early in training; the paper does not report such a measurement, so this is a testable prediction rather than a demonstrated fact.","The layer-wise lags in the paper's Table 5 hint that shallow layers tend to pick short lags and deep layers longer ones on speech tasks; one could test whether this pattern corresponds to the temporal receptive fields each layer needs.","Dropping the temperature schedule and instead penalizing the variance of the softmax output, as the paper lists as future work, could remove the decay-factor hyperparameter while preserving the hardmax limit."],"forward_implications":["SRC-SNN improves over vanilla RSNN on S-MNIST, PS-MNIST, SSC, and GSC by changing only the recurrent span, with no extra inference-time compute cost.","ASRC-SNN outperforms SRC-SNN on all four benchmarks and is less sensitive to the maximum skip span $T_\\lambda$ once it exceeds a threshold, easing hyperparameter search.","The improvement transfers across neuron models: SRC/ASRC consistently beat vanilla recurrent versions of PLIF, GLIF, CLIF, and TC-LIF, with ASRC usually ahead.","On the harder SCIFAR task with six layers, SRC-SNN gains more than 8% over RSNN and ASRC-SNN gains more than 5% over SRC-SNN, with more stable accuracy.","Because the learned final lags vary across layers and datasets, the data-dependent optimal span is something ASRC discovers automatically, whereas SRC needs it tuned per layer."],"supporting_citations":[{"why":"Supplies the adaptive-LIF recurrent baseline (ALIF) that the paper compares against and cites as the leading RSNN temporal model.","marker":"[18]"},{"why":"Supplies the TC-LIF baseline and the data preprocessing for GSC/SSC, and shares the triangle surrogate-gradient choice used in the analysis.","marker":"[21]"},{"why":"Provides the orthogonal initialization the paper uses to suppress the exploding-gradient branch of the temporal Jacobian.","marker":"[23]"},{"why":"Establishes surrogate-gradient learning as the differentiability mechanism for spike firing, which the gradient derivation relies on.","marker":"[37]"},{"why":"Provides the triangle surrogate function used in Eq. (6), the bound that makes the $|W_2|/V_{\\mathrm{th}}$ factor appear in Eq. (8).","marker":"[38]"},{"why":"Frames the vanishing/exploding gradient problem in RNNs that the paper adapts to the temporal dimension of spiking networks.","marker":"[39]"},{"why":"Prior analysis of skip recurrent connections in RNNs that the paper extends to spiking neurons, grounding the SRC design.","marker":"[40]"},{"why":"Supplies the low-temperature softmax asymptotics that justify annealing the skip-selection kernel to hardmax in ASRC.","marker":"[41]"},{"why":"Source of the Google Speech Commands dataset used as one of the four long-sequence benchmarks.","marker":"[50]"},{"why":"Source of the Heidelberg spiking speech commands (SSC) benchmark, the spike-based dataset used in the main evaluation.","marker":"[51]"}],"fun_headline_variants":["Adaptive skip connections fix vanishing gradients in spiking RNNs","Spiking RNNs learn adaptive skip lags to beat gradient vanishing","Adaptive skip recurrent connections fix vanishing temporal gradients","Skip recurrent connections rescue spiking memory from vanishing gradients","ASRC-SNN: adaptive skip lags for longer spiking memory"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported accuracy gains of SRC/ASRC over vanilla RSNNs reflect the skip-connection mechanism and not per-dataset hyperparameter tuning or single-run noise, since the paper reports no repeated-seed statistics and takes baselines from other papers with different setups.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive skip connections fix vanishing gradients in spiking RNNs","Spiking RNNs learn adaptive skip lags to beat gradient vanishing","Adaptive skip recurrent connections fix vanishing temporal gradients","Skip recurrent connections rescue spiking memory from vanishing gradients","ASRC-SNN: adaptive skip lags for longer spiking memory"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00093,"raw_usage":{"total_tokens":4044,"prompt_tokens":1066,"completion_tokens":2978,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":682,"completion_tokens_details":{"reasoning_tokens":2892}},"tokens_in":682,"tokens_out":2978,"duration_ms":20430,"temperature":1.0,"reasoning_tokens":2892,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:53:14.143258+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled re-run with identical hidden sizes, optimizers, epochs, and per-dataset hyperparameter search across many seeds would falsify the empirical claim if SRC/ASRC do not beat the vanilla RSNN outside seed noise; separately, measuring temporal gradient norms during training would falsify the mechanism claim if SRC/ASRC do not show larger temporal gradients than vanilla RSNN.","supporting_citations":[{"cited_title":"Tc-lif: A two-compartment spiking neuron model for long-term sequential modelling","cited_arxiv_id":null,"evidence_quote":"Supplies the TC-LIF baseline and the data preprocessing for GSC/SSC, and shares the triangle surrogate-gradient choice used in the analysis."},{"cited_title":"Recurrent orthogonal networks and long- memory tasks","cited_arxiv_id":null,"evidence_quote":"Provides the orthogonal initialization the paper uses to suppress the exploding-gradient branch of the temporal Jacobian."},{"cited_title":"Architectural complexity measures of recurrent neural networks","cited_arxiv_id":null,"evidence_quote":"Prior analysis of skip recurrent connections in RNNs that the paper extends to spiking neurons, grounding the SRC design."}],"review_version":1}