Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Block-Biased Mamba for Long-Range Sequence Processing

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mamba's S6 unit fails on long-range tasks for three structural reasons, and the paper's B2S6 unit fixes all three, beating S4 and S4D on Long-Range Arena while matching Mamba's language-modeling performance.

desk verdict Strong theory, promising architecture, but the headline empirical claim is unproven because the S6 baseline is causal while B2S6 is bidirectional. read the letter →

arxiv 2505.09022 v1 pith:N2C53SHL submitted 2025-05-13 cs.LG cs.AIstat.ML

Annan Yu , N. Benjamin Erichson This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords statespacemodelsMambaselectivespaceslong-rangesequencemodelingArenauniversalapproximationtrainingstabilityinductivebias
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that Mamba's weak performance on long-range sequence tasks is not accidental but follows from three structural defects in its S6 recurrent unit: the unit shares its input and output matrices across all channels, which caps its effective width and kills universal approximation; its input-dependent sampling interval produces an exponential sensitivity bias that lets large-magnitude inputs erase earlier context; and gradients with respect to the sampling-interval parameters grow faster than S4D's, making long-sequence training unstable. To fix this, the paper introduces B2S6, which partitions the input into blocks with independent selection weights and adds an input-independent, channel-specific bias term, and proves that B2S6 is a universal approximator and has only polynomial input-magnitude bias. On the Long-Range Arena benchmark, B2S6 reports an average accuracy of 87.71, above S4D's 84.89 and S4's 86.09, while the paper's S6 row averages 66.59. If these results hold, Mamba's long-range weakness is not inherent to selective recurrence; a modest architectural change can close most of the gap without giving up language-modeling perplexity.

What carries the argument

The machinery is the S6 recurrence $x^{(i)}_k = A^{(i)}_k x^{(i)}_{k-1} + B^{(i)}_k u^{(i)}_k$, $y_k = C_k x_k$, with $A^{(i)}_k=\exp(\Delta^{(i)}_k A)$, $\Delta^{(i)}_k=\mathrm{softplus}(w^\top u_k+b^{(i)})$, $B^{(i)}_k=A^{-1}(A^{(i)}_k-I)Bu_k$, and $C_k=u_k^\top C$. Its expressiveness defect is that $B$ and $C$ are shared across channels, making the final output a quadratic function of the input that cannot distinguish certain pairs of sequences. B2S6 modifies this by splitting $u_k$ into $h$ blocks of size $p=d/h$ and giving each block its own $B_{\mathrm{weight}}$, $B_{\mathrm{bias}}$, and $C$, so each block sees only its subvector but the collection of blocks behaves like a wider unit; the bias term $B^{(i)}_{\mathrm{bias}}$ is input-independent and channel-specific, providing additional effective width.

What would settle it

Take the exact Mamba/S6 configuration with bidirectional processing, complex parameters, and the same learning-rate schedule used for B2S6, and run it on all six Long-Range Arena tasks; if a properly tuned bidirectional S6 reaches or passes the 87.71 average that B2S6 reports, the paper's core claim that S6 fails on these tasks is refuted. A second check is to train single-layer S6 and S4D models on a continuous target with growing width: if S6's error falls with width in the same way as S4D's, then the non-universality theorem's practical significance fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that a single-layer S6 unit is not a universal approximator of continuous sequence-to-point functions, whereas a single-layer S4D unit is, and that this difference is caused by S6 sharing the matrices $B$ and $C$ across all channels. Beyond expressiveness, the paper proves that S6's input-dependent $\Delta$ imposes an exponentially large relative-gradient bias when any input vector grows in magnitude, and that the gradients of the S6 output with respect to $\Delta$ parameters are $\Omega(c^3)$ times S4D's as input magnitude $c\to\infty$ and grow relative to S4D as sequence length $L$ increases. The proposed B2S6 restores universal approximation either with the block structure alone or with the channel-specific bias alone, softens the bias from exponential to polynomial rates, and, together with a reduced learning rate on the $\Delta$ parameters, trains stably on long sequences. Empirically, B2S6 achieves an average Long-Range Arena accuracy of 87.71, making it, by the paper's account, the first selective SSM to reach state-of-the-art performance on this benchmark, while language-modeling perplexity on SlimPajama-6B stays close to S6's.

Load-bearing premise

The load-bearing premise is that the S6 baseline was trained comparably to B2S6, with similar bidirectional structure, normalization, and schedule, and the stability theorem's $w=0$ assumption means the training-instability diagnosis does not cover the input-dependent selection mechanism itself.

Editorial extensions

If this is right

  • Replacing only the recurrent unit in Mamba with B2S6 lifts average Long-Range Arena accuracy from 66.59 to 87.71, so the architecture outside the S6 unit is not the bottleneck.
  • Because a single S6 unit is not a universal approximator, any model built only from S6 units needs many layers or unshared channel parameters to fit continuous sequence functions; B2S6 removes that requirement at the unit level.
  • The reduced learning rate on the input-dependent sampling-interval parameters should be adopted in other long-sequence settings, since the gradient ratios in Theorem 3 grow with sequence length regardless of task.
  • On language modeling, B2S6's perplexity tracks S6 closely on SlimPajama-6B, so the long-range fix does not sacrifice the utility that makes Mamba attractive.
  • S6's exponential input-magnitude bias predicts that large-magnitude tokens can overwrite memory; B2S6's block split makes this failure unlikely because independent blocks with opposite selection signs cancel the extreme behavior.
  • The paper's comparison would be strengthened by a matched S6 baseline: the S6 row in Table 2 is reported without the configuration given for B2S6 in Appendix F, so a re-run with bidirectional S6, complex parameters, and the same schedule would test whether the reported gain comes from the B2S6 changes or from the training setup.
  • Theorem 3 fixes $w=0$, meaning the stability analysis holds only for the length-dependent part of the selective mechanism; the input-dependent part remains unanalyzed, so the reduced-learning-rate recommendation is an engineering remedy whose theoretical justification covers a restricted regime.
  • The block structure alone restores universal approximation, which suggests that multihead selective models already inherit part of the expressiveness fix; this isolates B2S6's distinctive contribution to the bias term and the learning-rate treatment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper analyzes why Mamba's S6 unit underperforms on Long-Range Arena (LRA) and proposes B2S6, a modification that partitions the input channels into blocks and adds a channel-specific bias term. The theoretical part claims three limitations of S6: single-layer S6 is not a universal approximator (Theorem 1), S6 imposes an exponentially input-magnitude-dependent sensitivity bias (Theorem 2), and S6 is less stable to train than S4D (Theorem 3). The paper then proves that B2S6 regains universality (Theorem 4) and exhibits a milder polynomial sensitivity (Theorem 5). Empirically, Table 2 reports an average LRA accuracy of 87.71 for B2S6 versus 66.59 for S6 (Mamba), and Appendix G reports that B2S6 roughly matches Mamba's perplexity on a SlimPajama subset.

Significance. If the empirical comparison were properly controlled, this paper would be a useful contribution: it identifies a concrete expressiveness bottleneck in S6, proposes a simple architectural repair, and backs the results with detailed proofs. The strengths are the self-contained Lemma 1 construction for the non-universality result, the explicit proof appendices for Theorems 1-5, the reproducible B2S6 configuration table in Appendix F, and the synthetic experiments that directly illustrate the theoretical statements. However, the headline LRA claim is currently confounded by protocol differences, and Theorem 3 explicitly fixes w=0, so the paper's central explanation of Mamba's LRA failure is not fully established. These issues substantially limit the significance of the present version.

major comments (4)
  1. [Section 7, Table 2; Appendix A; Appendix F] The central empirical claim that B2S6 "resurrects Mamba from failure" on LRA is not established because the S6 baseline is not compared under a matched protocol. Appendix A states that the authors adopt a bidirectional structure in training the LRA tasks, and Appendix F (Table 4) provides full configurations for B2S6 but no configuration for the S6 row. B2S6 also differs from the published S6 model by using complex-valued parameters, normalization choices, and a reduced learning rate for Delta-related parameters. Since bidirectionality and these training choices are known to affect LRA accuracy substantially, the 21-point gap between S6 (66.59) and B2S6 (87.71) cannot be attributed to the proposed block partition and channel bias alone. The authors should either train a same-protocol S6 baseline, or ablate bidirectionality, complex parameterization, and the reduced learning rate to isolate the effect of the proposed mechanism.
  2. [Section 5, Theorem 3] Theorem 3 fixes w=0, which makes the sampling interval Delta_k = softplus(b) input-independent. The paper itself acknowledges this at the end of Section 5: "We also fixed w = 0, leaving the case when w≠0 for future work." Consequently, Theorem 3 does not analyze the input-dependent selection mechanism that abstract and Section 1 identify as the source of instability. The statement "this instability comes from the input-dependent selection mechanism" is therefore not supported by the theorem. The theorem should be reframed as a statement about S6's shared B/C structure under a fixed sampling interval, or extended to the selective w≠0 case before the paper can claim to explain the role of selectivity in training instability.
  3. [Section 7, Table 3] The paper claims in Section 1 and Section 7 that B2S6 achieves "state-of-the-art performance" on LRA, but the extended Table 3 directly contradicts this: HOPE-SSM (87.85) and S4D-FT (88.26) have higher average accuracies, and RTF SSM ties B2S6 at 87.71. The claim should be revised to "competitive with published SSMs" or restricted to selective SSMs, for which the claim may still hold. This is a factual inconsistency in a stated contribution and should be corrected.
  4. [Section 4, Eq. (6); Theorems 2 and 5] The paper interprets the relative gradient sensitivity S_k as a measure of inductive bias and memory retention, but S_k as defined in Eq. (6) measures the sensitivity of the final output to perturbations of each input, not whether information is actually stored or discarded by the recurrence. The statements that S6 "can cause the model to discard useful long-term information too quickly" and that B2S6 "preserves memory" rely on this identification. The authors should either provide direct evidence connecting S_k to long-range memory behavior (e.g., probing or reconstruction tasks), or restrict the claims to gradient sensitivity rather than memory retention.
minor comments (6)
  1. [Section 2, last paragraph] The sentence "In the next section, show how this distinction limits the 'effective width' and expressiveness of an S6 unit" is missing a grammatical subject; it should be "we show how".
  2. [References [87] and [88]] References [87] and [88] cite the same paper by Wang and Xue; one of them should be removed or the two citations should be distinguished by their specific claims.
  3. [Figure 1 caption] The caption uses the block size p and number of blocks h before these are defined in Section 6; a brief parenthetical definition in the caption would improve readability.
  4. [Section 6, paragraph after Theorem 5] The sentence "the probabilities that a given input satisfies jw^T juk0 < 0 and jw^T juk0 > 0 are equal" refers to random initialization of w; this should be stated explicitly in the main text rather than implied, since the probability statement is over initialization, not over inputs.
  5. [Appendix F, Table 4] The hyperparameter h is described as "not carefully fine-tuned but rather picked randomly"; reporting a small sensitivity study over h, or at least stating the range tested, would strengthen the empirical claims.
  6. [Appendix G] The language-modeling experiment reports perplexity only up to 28,830 steps and does not provide final validation perplexity, model size per variant, or token count; a one-epoch preliminary result is acceptable, but the main text should describe it as preliminary with these limitations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: theoretical results are self-contained; the LRA baseline asymmetry is a correctness risk, not a definitional reduction.

full rationale

The paper's derivation chain is not circular. Theorem 1 is proved directly from the model equations: the S6 output is reduced to a quadratic encoder and Lemma 1 exhibits a continuous function that no quadratic function can approximate, while the S4D positive result is built on the external Pinkus and Wang-Xue UATs. Theorem 4 reduces B2S6 universality to the S4D construction, and Theorems 2 and 5 compute explicit Jacobian asymptotics from the paper's own recurrence equations rather than assuming their conclusions. The sensitivity statistic S_k in eq. (6) is a defined measure; the interpretive claim that this measure tracks long-range-task suitability is an empirical assumption corroborated by synthetic experiments, not a definitional identity. Theorem 3 is an analytic comparison under the stated assumption w=0, and while this limits the theorem's reach relative to the paper's narrative about the selection mechanism, it is a correctness risk rather than circularity. The empirical LRA claim is also not circular: the S4/S4D rows are external published results, and B2S6 is a genuinely different trained model. However, Appendix F provides configurations for B2S6 but not for the S6 baseline, and Appendix A states that the authors adopt bidirectional structure for their LRA training, so the 21-point gap over the published causal Mamba row may be partly attributable to protocol differences (bidirectionality, complex parameterization, reduced Delta learning rate). That is a confound in the headline comparison, not a reduction of the prediction to its inputs. The only self-citations are minor and non-load-bearing, e.g., [98] is cited for the elementary decomposition of a diagonal SSM into scalar systems, a fact also supported by external results; the central UAT and non-UAT arguments do not depend on the authors' own prior work.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central results rely on standard approximation theorems for MLPs and LTI systems, on the authors' S_k sensitivity measure, on the simplification w=0 in the stability theorem, and on the comparability of the S6 LRA baseline. No new physical entities are introduced.

free parameters (2)
  • Number of blocks h = 8
    Set to h=8 for all LRA tasks with p=#Features/8; the authors state it was "not carefully fine-tuned but rather picked randomly" (Appendix F). This is a hand-chosen hyperparameter that controls the block-wise selective structure.
  • Learning-rate multiplier for Delta-related parameters = not stated
    The paper reduces the learning rate of jw and jb(i) to improve training stability (Section 6), but the exact multiplier is not reported. This is a hand-chosen training choice that affects the claim of stability.
assumptions (6)
  • standard math A two-layer wide network with a Lipschitz, non-polynomial activation is a universal approximator of continuous functions (Pinkus).
    Used in the proof of Theorem 1 Part I; cited as [60].
  • standard math LTI state space models with fixed discretization can approximate arbitrary convolutional kernels (Wang and Xue).
    Used in the proof of Theorem 1 Part I to realize the weights of the MLP as an S4D system; cited as [87, 88].
  • domain assumption A diagonal state matrix A with negative entries suffices to represent the S4D and S6 dynamics analyzed here.
    Mamba and S4D use diagonal A in practice; the paper also ties A across channels in the LRA setup (Section 2 footnote).
  • ad hoc to paper The relative-gradient sensitivity S_k defined in eq. (6) is a meaningful measure of inductive bias for long-range sequence tasks.
    The paper introduces S_k to conclude that S6 has an exponential bias and B2S6 has a milder one; no external benchmark validates that S_k correlates with LRA performance.
  • ad hoc to paper Theorem 3 fixes w=0, making S6's sampling interval constant and input-independent.
    The paper acknowledges this assumption (Section 5: 'We also fixed w=0, leaving the case when w does not equal 0 for future work'), but the conclusion about the input-dependent selection mechanism is not derived in the w not equal to 0 regime.
  • domain assumption The 'S6 (Mamba)' baseline in Table 2 was trained under conditions comparable to B2S6.
    Appendix F provides B2S6 configurations but no S6 baseline configuration; if the baseline used a different protocol, the comparison is unfair.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Block-Biased Mamba for Long-Range Sequence Processing." pith.science (2026). https://pith.science/paper/N2C53SHL

@misc{pith2026250509022,
  author       = {Pith},
  title        = {Pith review of: Block-Biased Mamba for Long-Range Sequence Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N2C53SHL}},
  note         = {Machine review of arXiv:2505.09022}
}
abstract

Mamba extends earlier state space models (SSMs) by introducing input-dependent dynamics, and has demonstrated strong empirical performance across a range of domains, including language modeling, computer vision, and foundation models. However, a surprising weakness remains: despite being built on architectures designed for long-range dependencies, Mamba performs poorly on long-range sequential tasks. Understanding and addressing this gap is important for improving Mamba's universality and versatility. In this work, we analyze Mamba's limitations through three perspectives: expressiveness, inductive bias, and training stability. Our theoretical results show how Mamba falls short in each of these aspects compared to earlier SSMs such as S4D. To address these issues, we propose $\text{B}_2\text{S}_6$, a simple extension of Mamba's S6 unit that combines block-wise selective dynamics with a channel-specific bias. We prove that these changes equip the model with a better-suited inductive bias and improve its expressiveness and stability. Empirically, $\text{B}_2\text{S}_6$ outperforms S4 and S4D on Long-Range Arena (LRA) tasks while maintaining Mamba's performance on language modeling benchmarks.

Figures

Figures reproduced from arXiv: 2505.09022 by the authors.

Figure 1
Figure 1. Comparison of S4D, S6, and B2S6 units. S4D uses independent linear SSM units for each channel, giving it high capacity (or width) but no input-dependent selectivity. S6 introduces a selective mechanism that modulates its internal dynamics based on the input, but shares parameters across channels, limiting its effective width and expressiveness. Our proposed B2S6 unit partitions the input into smaller blocks, enablin… view at source ↗
Figure 2
Figure 2. The architecture of the neural network eq. ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The mean loss ∥g˜ − g∥2 between the true coefficient g and the model prediction g˜. Every model has a single layer and is trained for 10 epochs. Here, d is the number of channels in a model. For the B2S6 model, h is the number of blocks and p is the number of channels in each block. We present a simple toy experiment to illustrate Theorem 1. Consider the input function u(t; g) = P10 i=1 gi cos(pit), where p1, . . . … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The mean relative loss |G(u) − G˜(u)|/σ1 for different choices of σ1 and σ2. The S6 model cannot make useful predictions when σ1 is small and σ2 is large; B2S6 fixes this. In all experiments, we fix d = 32. For B2S6, we set h = 8 and p = 4. poor generalization refers t…
Figure 5
Figure 5. Figure 5: Numerical experiments to verify Theorem 3, where we compute the ratio between the gradients with respect to the S6 parameters and S4D parameters. For the first two figures, we fix L = 100; for the last figure, we fix c = 1. The gradients are computed using closed algeb…
Figure 6
Figure 6. Figure 6: The root-mean-squared loss ∥G(u) − G˜(u)∥2 between the true output G(u) and the model prediction G˜(u). The first two models are trained with a learning rate of 0.001 on the ∆ parameters, whereas the last model is trained with no training of the ∆ parameters. F Details…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Tracking

    cs.LG 2026-02 conditional novelty 6.0 of 10

    KLA reparameterises information-form Kalman filtering as an associative Möbius scan, giving a parallelisable nonlinear state-space mixer that tracks A5 permutations with one or two layers.

  2. Rethinking the long-range dependency in Mamba/SSM and transformer models

    cs.LG 2025-09 reject novelty 3.0 of 10

    SSM/Mamba long-range dependency decays exponentially with the time gap by construction; a proposed interaction-based hidden state update can break this decay, but its proven stability covers only a restrictive special case.

Reference graph

Works this paper leans on

108 extracted references · 42 canonical work pages · cited by 2 Pith papers

  1. [1]

    Spectral state space models

    Naman Agarwal, Daniel Suo, Xinyi Chen, and Elad Hazan. Spectral state space models. arXiv preprint arXiv:2312.06837 , 2023

    arXiv 2023

  2. [2]

    Learning and generalization in overpa- rameterized neural networks, going beyond two layers

    Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang. Learning and generalization in overpa- rameterized neural networks, going beyond two layers. Advances in neural information processing systems, 32, 2019

    2019

  3. [3]

    State space models as foundation models: A control theoretic overview

    Carmen Amo Alonso, Jerome Sieber, and Melanie N Zeilinger. State space models as foundation models: A control theoretic overview. arXiv preprint arXiv:2403.16899 , 2024

    arXiv 2024

  4. [4]

    Towards better under- standing of gradient-based attribution methods for deep neural networks

    Marco Ancona, Enea Ceolini, Cengiz ¨Oztireli, and Markus Gross. Towards better under- standing of gradient-based attribution methods for deep neural networks. arXiv preprint arXiv:1711.06104, 2017

    arXiv 2017

  5. [5]

    Intrinsic dimension of data representations in deep neural networks

    Alessio Ansuini, Alessandro Laio, Jakob H Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. Advances in Neural Information Processing Systems, 32, 2019

    2019

  6. [6]

    Unitary evolution recurrent neural networks

    Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary evolution recurrent neural networks. In International Conference on Machine Learning , pages 1120–1128. PMLR, 2016

    2016

  7. [7]

    Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks

    Sanjeev Arora, Simon Du, Wei Hu, Zhiyuan Li, and Ruosong Wang. Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks. In International conference on machine learning , pages 322–332. PMLR, 2019

    2019

  8. [8]

    How to explain individual classification decisions

    David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert M¨ uller. How to explain individual classification decisions. The Journal of Machine Learning Research, 11:1803–1831, 2010

    2010

Show all 108 references
  1. [9]

    An empirical evaluation of generic convo- lutional and recurrent networks for sequence modeling

    Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convo- lutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 , 2018

    2018 arXiv

  2. [10]

    Deep learning: a statistical viewpoint

    Peter L Bartlett, Andrea Montanari, and Alexander Rakhlin. Deep learning: a statistical viewpoint. Acta numerica, 30:87–201, 2021. 13

    2021

  3. [11]

    The convergence rate of neural networks for learned functions of different frequencies

    Ronen Basri, David Jacobs, Yoni Kasten, and Shira Kritchman. The convergence rate of neural networks for learned functions of different frequencies. Adv. Neur. Info. Proc. Syst. , 32, 2019

    2019

  4. [12]

    Fast convolution algorithm for state space models

    Gregory Beylkin. Fast convolution algorithm for state space models. arXiv preprint arXiv:2411.17729, 2024

    2024 arXiv

  5. [13]

    Antisymmetricrnn: A dynamical system view on recurrent neural networks

    Bo Chang, Minmin Chen, Eldad Haber, and Ed H Chi. Antisymmetricrnn: A dynamical system view on recurrent neural networks. In International Conference on Machine Learning, 2019

    2019

  6. [14]

    Entropy-sgd: Biasing gradient descent into wide valleys

    Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, and Riccardo Zecchina. Entropy-sgd: Biasing gradient descent into wide valleys. Journal of Statistical Mechanics: Theory and Experiment, 2019(12):124...

    2019

  7. [15]

    Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems

    Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks , 6(4):911–917, 1995

    1995

  8. [16]

    The loss surfaces of multilayer networks

    Anna Choromanska, Mikael Henaff, Michael Mathieu, G´ erard Ben Arous, and Yann LeCun. The loss surfaces of multilayer networks. In Artificial intelligence and statistics , pages 192–204. PMLR, 2015

    2015

  9. [17]

    Rethinking attention with performers

    Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. Rethinking attention with performers. In International Conference on Machine Learning , 2020

    2020

  10. [18]

    Approximation by superpositions of a sigmoidal function

    George Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems , 2(4):303–314, 1989

    1989

  11. [19]

    Self-stabilization: The implicit bias of gra- dient descent at the edge of stability.International Conference on Learning Representations, 2023

    Alex Damian, Eshaan Nichani, and Jason D Lee. Self-stabilization: The implicit bias of gra- dient descent at the edge of stability.International Conference on Learning Representations, 2023

    2023

  12. [20]

    Transformers are ssms: Generalized models and efficient algorithms through structured state space duality

    Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.21060 , 2024

    2024 arXiv

  13. [21]

    Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

    Yann N Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. Advances in neural information processing systems , 27, 2014

    2014

  14. [22]

    Fast fourier transforms for nonequispaced data

    Alok Dutt and Vladimir Rokhlin. Fast fourier transforms for nonequispaced data. SIAM Journal on Scientific computing , 14(6):1368–1393, 1993

    1993

  15. [23]

    Lipschitz recurrent neural networks

    N Benjamin Erichson, Omri Azencot, Alejandro Queiruga, Liam Hodgkinson, and Michael W Mahoney. Lipschitz recurrent neural networks. In International Conference on Learning Representations, 2021

    2021

  16. [24]

    Understanding the difficulty of training deep feedforward neural networks

    Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics , pages 249–256. JMLR Workshop and Conference Proceedings, 2010

    2010

  17. [25]

    Unlocking state-tracking in linear rnns through negative eigenvalues

    Riccardo Grazzi, Julien Siems, Arber Zela, J¨ org KH Franke, Frank Hutter, and Mas- similiano Pontil. Unlocking state-tracking in linear rnns through negative eigenvalues. International Conference on Learning Representations, 2025. 14

    2025

  18. [26]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 , 2023

    2023 arXiv

  19. [27]

    On the parameterization and initialization of diagonal state space models

    Albert Gu, Karan Goel, Ankit Gupta, and Christopher R´ e. On the parameterization and initialization of diagonal state space models. Advances in Neural Information Processing Systems, 35:35971–35983, 2022

    2022

  20. [28]

    Efficiently modeling long sequences with structured state spaces

    Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations, 2022

    2022

  21. [29]

    Diagonal state spaces are as effective as structured state spaces

    Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are as effective as structured state spaces. Advances in Neural Information Processing Systems , 35:22982– 22994, 2022

    2022

  22. [30]

    Liquid structural state-space models

    Ramin Hasani, Mathias Lechner, Tsun-Hsuan Wang, Makram Chahine, Alexander Amini, and Daniela Rus. Liquid structural state-space models. International Conference on Learning Representations, 2023

    2023

  23. [31]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  24. [32]

    Generalization error analysis for selective state-space models through the lens of attention

    Arya Honarpisheh, Mustafa Bozdag, Mario Sznaier, and Octavia Camps. Generalization error analysis for selective state-space models through the lens of attention. arXiv preprint arXiv:2502.01473, 2025

    2025

  25. [33]

    State-space models are accurate and efficient neural operators for dynamical systems

    Zheyuan Hu, Nazanin Ahmadi Daryakenari, Qianli Shen, Kenji Kawaguchi, and George Em Karniadakis. State-space models are accurate and efficient neural operators for dynamical systems. arXiv preprint arXiv:2409.03231 , 2024

    2024 arXiv

  26. [34]

    Hydra: Bidirectional state space models through generalized matrix mixers

    Sukjun Hwang, Aakash Sunil Lahoti, Ratish Puduppully, Tri Dao, and Albert Gu. Hydra: Bidirectional state space models through generalized matrix mixers. Advances in Neural Information Processing Systems, 37:110876–110908, 2024

    2024

  27. [35]

    Jacot, F

    A. Jacot, F. Gabriel, and C. Hongler. Neural tangent kernel: Convergence and generaliza- tion in neural networks. Adv. Neur. Info. Proc. Syst. , 31, 2018

    2018

  28. [36]

    A new approach to linear filtering and prediction problems

    Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. J. Basic Eng, 82(1):35–45, 1960

    1960

  29. [37]

    Transformers are rnns: Fast autoregressive transformers with linear attention

    Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and Fran¸ cois Fleuret. Transformers are rnns: Fast autoregressive transformers with linear attention. In International conference on machine learning , pages 5156–5165. PMLR, 2020

    2020

  30. [38]

    On large-batch training for deep learning: Generalization gap and sharp minima

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. International Conference on Learning Representations, 2017

    2017

  31. [39]

    Patrick Kidger and Terry J. Lyons. Universal approximation with deep narrow networks. CoRR, abs/1905.08539, 2019

    1905 arXiv

  32. [40]

    Exploring the loss landscape of regularized neural networks via convex duality

    Sungyoon Kim, Aaron Mishkin, and Mert Pilanci. Exploring the loss landscape of regularized neural networks via convex duality. International Conference on Learning Representations, 2025

    2025

  33. [41]

    Reformer: The efficient transformer

    Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In International Conference on Machine Learning , 2020. 15

    2020

  34. [42]

    Visualizing the loss landscape of neural nets

    Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Advances in neural information processing systems , 31, 2018

    2018

  35. [43]

    Spmamba: State-space model is all you need in speech separation

    Kai Li, Guo Chen, Runxuan Yang, and Xiaolin Hu. Spmamba: State-space model is all you need in speech separation. arXiv preprint arXiv:2404.02063 , 2024

    2024 arXiv

  36. [44]

    Videomamba: State space model for efficient video understanding

    Kunchang Li, Xinhao Li, Yi Wang, Yinan He, Yali Wang, Limin Wang, and Yu Qiao. Videomamba: State space model for efficient video understanding. In European Conference on Computer Vision , pages 237–255. Springer, 2024

    2024

  37. [45]

    Jamba: A hybrid transformer-mamba language model

    Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, et al. Jamba: A hybrid transformer-mamba language model. International Conference on Learning Representations, 2025

    2025

  38. [46]

    From generalization analysis to optimization designs for state space models

    Fusheng Liu and Qianxiao Li. From generalization analysis to optimization designs for state space models. arXiv preprint arXiv:2405.02670 , 2024

    2024 arXiv

  39. [47]

    Autocorrelation matters: Understanding the role of initialization schemes for state space models

    Fusheng Liu and Qianxiao Li. Autocorrelation matters: Understanding the role of initialization schemes for state space models. International Conference on Learning Representations, 2025

    2025

  40. [48]

    Sigma: Selective gated mamba for sequential recommenda- tion

    Ziwei Liu, Qidong Liu, Yejing Wang, Wanyu Wang, Pengyue Jia, Maolin Wang, Zitao Liu, Yi Chang, and Xiangyu Zhao. Sigma: Selective gated mamba for sequential recommenda- tion. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 39, pages 12264–12272, 2025

    2025

  41. [49]

    Sgdr: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. International Conference on Learning Representations, 2017

    2017

  42. [50]

    Learning nonlinear operators via deeponet based on the universal approximation theorem of operators

    Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence , 3(3):218–229, 2021

    2021

  43. [51]

    A mamba foundation model for time series forecasting

    Haoyu Ma, Yushu Chen, Wenlai Zhao, Jinzhe Yang, Yingsheng Ji, Xinghua Xu, Xiaozhu Liu, Hao Jing, Shengzhuo Liu, and Guangwen Yang. A mamba foundation model for time series forecasting. arXiv preprint arXiv:2411.02941 , 2024

    2024 arXiv

  44. [52]

    A mean field view of the landscape of two-layer neural networks.Proceedings of the National Academy of Sciences, 115(33):E7665– E7671, 2018

    Song Mei, Andrea Montanari, and Phan-Minh Nguyen. A mean field view of the landscape of two-layer neural networks.Proceedings of the National Academy of Sciences, 115(33):E7665– E7671, 2018

    2018

  45. [53]

    Theoretical foundations of deep selective state-space models

    Nicola Muca Cirone, Antonio Orvieto, Benjamin Walker, Cristopher Salvi, and Terry Lyons. Theoretical foundations of deep selective state-space models. Advances in Neural Information Processing Systems, 37:127226–127272, 2024

    2024

  46. [54]

    A time series is worth 64 words: Long-term forecasting with transformers

    Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023

    2023

  47. [55]

    Resurrecting recurrent neural networks for long sequences

    Antonio Orvieto, Samuel L Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De. Resurrecting recurrent neural networks for long sequences. arXiv preprint arXiv:2303.06349, 2023

    2023 arXiv

  48. [56]

    State-free inference of state-space models: The transfer function approach

    Rom N Parnichkun, Stefano Massaroli, Alessandro Moro, Jimmy TH Smith, Ramin Hasani, Mathias Lechner, Qi An, Christopher R´ e, Hajime Asama, and Stefano Ermon. State-free inference of state-space models: The transfer function approach. International Conference on Machine Learni...

    2024

  49. [57]

    On the difficulty of training recurrent neural networks

    Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. In International conference on machine learning , pages 1310–1318. Pmlr, 2013

    2013

  50. [58]

    Mamba-360: Survey of state space models as transformer alternative for long sequence modelling: Methods, applications, and challenges

    Badri Narayana Patro and Vijay Srinivas Agneeswaran. Mamba-360: Survey of state space models as transformer alternative for long sequence modelling: Methods, applications, and challenges. arXiv preprint arXiv:2404.16112 , 2024

    2024 arXiv

  51. [59]

    Let SSMs be ConvNets: State-space modeling with optimal tensor contractions

    Yan Ru Pei. Let SSMs be ConvNets: State-space modeling with optimal tensor contractions. International Conference on Learning Representations, 2025

    2025

  52. [60]

    Approximation theory of the mlp model in neural networks

    Allan Pinkus. Approximation theory of the mlp model in neural networks. Acta numerica, 8:143–195, 1999

    1999

  53. [61]

    S4++: Elevating long sequence modeling with state memory reply

    Biqing Qi, Junqi Gao, Dong Li, Kaiyan Zhang, Jianxing Liu, Ligang Wu, and Bowen Zhou. S4++: Elevating long sequence modeling with state memory reply. 2024

    2024

  54. [62]

    Vl-mamba: Exploring state space models for multimodal learning

    Yanyuan Qiao, Zheng Yu, Longteng Guo, Sihan Chen, Zijia Zhao, Mingzhen Sun, Qi Wu, and Jing Liu. Vl-mamba: Exploring state space models for multimodal learning. arXiv preprint arXiv:2403.13600, 2024

    2024 arXiv

  55. [63]

    Per- turbed state space feature encoders for optical flow with event cameras

    Gokul Raju Govinda Raju, Nikola Zubi´ c, Marco Cannici, and Davide Scaramuzza. Per- turbed state space feature encoders for optical flow with event cameras. arXiv preprint arXiv:2504.10669, 2025

    2025 arXiv

  56. [64]

    Provable benefits of complex parameterizations for structured state space models

    Yuval Ran-Milo, Eden Lumbroso, Edo Cohen-Karlik, Raja Giryes, Amir Globerson, and Nadav Cohen. Provable benefits of complex parameterizations for structured state space models. Advances in Neural Information Processing Systems , 37:115906–115939, 2024

    2024

  57. [65]

    Serpent: Selective resampling for expressive state space models

    Stefano Rando, Luca Romani, Matteo Migliarini, Luca Franco, Denis Gudovskiy, and Fabio Galasso. Serpent: Selective resampling for expressive state space models. arXiv preprint arXiv:2501.11729, 2025

    2025 arXiv

  58. [66]

    Ckconv: Continuous kernel convolution for sequential data

    David W Romero, Anna Kuzina, Erik J Bekkers, Jakub M Tomczak, and Mark Hoogen- doorn. Ckconv: Continuous kernel convolution for sequential data. In International Conference on Machine Learning, 2022

    2022

  59. [67]

    Trainability and accuracy of artificial neural networks: An interacting particle system approach

    Grant Rotskoff and Eric Vanden-Eijnden. Trainability and accuracy of artificial neural networks: An interacting particle system approach. Communications on Pure and Applied Mathematics, 75(9):1889–1935, 2022

    1935

  60. [68]

    Unicornn: A recurrent model for learning very long time dependencies

    T Konstantin Rusch and Siddhartha Mishra. Unicornn: A recurrent model for learning very long time dependencies. In International Conference on Machine Learning , pages 9168–9178. PMLR, 2021

    2021

  61. [69]

    Oscillatory state-space models

    T Konstantin Rusch and Daniela Rus. Oscillatory state-space models. International Conference on Learning Representations, 2025

    2025

  62. [70]

    Empirical anal- ysis of the hessian of over-parametrized neural networks

    Levent Sagun, Utku Evci, V Ugur Guney, Yann Dauphin, and Leon Bottou. Empirical anal- ysis of the hessian of over-parametrized neural networks. arXiv preprint arXiv:1706.04454 , 2017

    2017 arXiv

  63. [71]

    Recurrent neural networks are universal approximators

    Anton Maximilian Sch¨ afer and Hans Georg Zimmermann. Recurrent neural networks are universal approximators. In Artificial Neural Networks–ICANN 2006: 16th International Conference, Athens, Greece, September 10-14, 2006. Proceedings, Part I 16 , pages 632–640. Springer, 2006

    2006

  64. [72]

    SpikingSSMs: Learning long sequences with sparse and 17 parallel spiking state space models

    Shuaijie Shen, Chao Wang, Renzhuo Huang, Yan Zhong, Qinghai Guo, Zhichao Lu, Jianguo Zhang, and Luziwei Leng. SpikingSSMs: Learning long sequences with sparse and 17 parallel spiking state space models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 3...

    2025

  65. [73]

    Learning important features through propagating activation differences

    Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. InInternational conference on machine learning, pages 3145–3153. PMlR, 2017

    2017

  66. [74]

    Understanding the differences in foundation models: Attention, state space models, and recurrent neural networks

    Jerome Sieber, Carmen Amo Alonso, Alexandre Didier, Melanie Zeilinger, and Antonio Orvieto. Understanding the differences in foundation models: Attention, state space models, and recurrent neural networks. Advances in Neural Information Processing Systems, 37:134534–134566, 2024

    2024

  67. [75]

    Deep inside convolutional net- works: Visualising image classification models and saliency maps

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional net- works: Visualising image classification models and saliency maps. International Conference on Learning Representations, 2014

    2014

  68. [76]

    Towards a theory of learning dynamics in deep state space models

    Jakub Sm´ ekal, Jimmy TH Smith, Michael Kleinman, Dan Biderman, and Scott W Linder- man. Towards a theory of learning dynamics in deep state space models. arXiv preprint arXiv:2407.07279, 2024

    2024 arXiv

  69. [77]

    Smith, Andrew Warrington, and Scott Linderman

    Jimmy T.H. Smith, Andrew Warrington, and Scott Linderman. Simplified state space layers for sequence modeling. In The Eleventh International Conference on Learning Representations, 2023

    2023

  70. [78]

    SlimPajama: A 627B token cleaned and deduplicated version of RedPajama, 2023

    Daria Soboleva, Faisal Al-Khateeb, Robert Myers, Jacob R Steeves, Joel Hestness, and Nolan Dey. SlimPajama: A 627B token cleaned and deduplicated version of RedPajama, 2023

    2023

  71. [79]

    S7: Selective and simplified state space layers for sequence modeling

    Taylan Soydan, Nikola Zubi´ c, Nico Messikommer, Siddhartha Mishra, and Davide Scara- muzza. S7: Selective and simplified state space layers for sequence modeling. arXiv preprint arXiv:2410.03464, 2024

    2024 arXiv

  72. [80]

    A survey on statistical theory of deep learning: Approxi- mation, training dynamics, and generative models

    Namjoon Suh and Guang Cheng. A survey on statistical theory of deep learning: Approxi- mation, training dynamics, and generative models. Annual Review of Statistics and Its Application, 12, 2024

    2024

  73. [81]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning , pages 3319–3328. PMLR, 2017

    2017

  74. [82]

    Benefits of depth in neural networks

    Matus Telgarsky. Benefits of depth in neural networks. In Conference on learning theory, pages 1517–1539. PMLR, 2016

    2016

  75. [83]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  76. [84]

    An empirical study of mamba-based language models

    Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, et al. An empirical study of mamba-based language models. arXiv preprint arXiv:2406.07887 , 2024

    2024 arXiv

  77. [85]

    Star-rec: Making peace with length variance and pattern diversity in sequential recommendation

    Maolin Wang, Sheng Zhang, Ruocheng Guo, Wanyu Wang, Xuetao Wei, Zitao Liu, Hongzhi Yin, Yi Chang, and Xiangyu Zhao. Star-rec: Making peace with length variance and pattern diversity in sequential recommendation. arXiv preprint arXiv:2505.03484 , 2025

    2025 arXiv

  78. [86]

    StableSSM: Alleviating the curse of memory in state-space models through stable reparameterization

    Shida Wang and Qianxiao Li. StableSSM: Alleviating the curse of memory in state-space models through stable reparameterization. arXiv preprint arXiv:2311.14495 , 2023. 18

    2023 arXiv

  79. [88]

    State-space models with layer-wise nonlinearity are universal approximators with exponential decaying memory

    Shida Wang and Beichen Xue. State-space models with layer-wise nonlinearity are universal approximators with exponential decaying memory. Advances in Neural Information Processing Systems, 36, 2024

    2024

  80. [89]

    State space model for new-generation network alternative to transformers: A survey

    Xiao Wang, Shiao Wang, Yuhe Ding, Yuehang Li, Wentao Wu, Yao Rong, Weizhe Kong, Ju Huang, Shihao Li, Haoxiang Yang, et al. State space model for new-generation network alternative to transformers: A survey. arXiv preprint arXiv:2404.09516 , 2024

    2024 arXiv

  81. [90]

    A deep state space model for rainfall-runoff simulations

    Yihan Wang, Lujun Zhang, Annan Yu, N Benjamin Erichson, and Tiantian Yang. A deep state space model for rainfall-runoff simulations. arXiv preprint arXiv:2501.14980 , 2025

    2025 arXiv

  82. [91]

    Is mamba effective for time series forecasting? Neurocomputing, 619:129178, 2025

    Zihan Wang, Fanheng Kong, Shi Feng, Ming Wang, Xiaocui Yang, Han Zhao, Daling Wang, and Yifei Zhang. Is mamba effective for time series forecasting? Neurocomputing, 619:129178, 2025

    2025

  83. [92]

    A superfast direct inversion method for the nonuniform discrete fourier transform

    Heather Wilber, Ethan N Epperly, and Alex H Barnett. A superfast direct inversion method for the nonuniform discrete fourier transform. arXiv e-prints, pages arXiv–2404, 2024

    2024

  84. [93]

    Disentangling trainability and generalization in deep neural networks

    Lechao Xiao, Jeffrey Pennington, and Samuel Schoenholz. Disentangling trainability and generalization in deep neural networks. In International Conference on Machine Learning , pages 10462–10472. PMLR, 2020

    2020

  85. [94]

    Evaluating loss landscapes from a topology perspective

    Tiankai Xie, Caleb Geniesse, Jiaqing Chen, Yaoqing Yang, Dmitriy Morozov, Michael W Mahoney, Ross Maciejewski, and Gunther H Weber. Evaluating loss landscapes from a topology perspective. arXiv preprint arXiv:2411.09807 , 2024

    2024 arXiv

  86. [95]

    Feature learning in infinite-width neural networks

    Greg Yang and Edward J Hu. Feature learning in infinite-width neural networks. arXiv preprint arXiv:2011.14522, 2020

    2011 arXiv

  87. [96]

    Longmamba: Enhancing mamba’s long-context capabilities via training-free receptive field enlargement

    Zhifan Ye, Kejing Xia, Yonggan Fu, Xin Dong, Jihoon Hong, Xiangchi Yuan, Shizhe Diao, Jan Kautz, Pavlo Molchanov, and Yingyan Celine Lin. Longmamba: Enhancing mamba’s long-context capabilities via training-free receptive field enlargement. In The Thirteenth International Confe...

    2025

  88. [97]

    Arbitrary-depth universal approximation theorems for operator neural networks

    Annan Yu, Chlo´ e Becquey, Diana Halikias, Matthew Esmaili Mallory, and Alex Townsend. Arbitrary-depth universal approximation theorems for operator neural networks. arXiv preprint arXiv:2109.11354, 2021

    2021 arXiv

  89. [98]

    Tuning frequency bias of state space models

    Annan Yu, Dongwei Lyu, Soon Hoe Lim, Michael W Mahoney, and N Benjamin Erichson. Tuning frequency bias of state space models. International Conference on Learning Representations, 2025

    2025

  90. [99]

    HOPE for a robust pa- rameterization of long-memory state space models

    Annan Yu, Michael W Mahoney, and N Benjamin Erichson. HOPE for a robust pa- rameterization of long-memory state space models. Internation Conference on Learning Representations, 2025

    2025

  91. [100]

    Mahoney, and N

    Annan Yu, Arnur Nigmetov, Dmitriy Morozov, Michael W. Mahoney, and N. Benjamin Erichson. Robustifying state-space models for long sequences via approximate diagonaliza- tion. In The Twelfth International Conference on Learning Representations , 2024

    2024

  92. [101]

    On the stability of unevenly spaced samples for interpolation and quadrature

    Annan Yu and Alex Townsend. On the stability of unevenly spaced samples for interpolation and quadrature. BIT Numerical Mathematics , 63(2):23, 2023. 19

    2023

  93. [102]

    Tuning frequency bias in neural network training with nonuniform data

    Annan Yu, Yunan Yang, and Alex Townsend. Tuning frequency bias in neural network training with nonuniform data. International Conference on Learning Representations , 2023

    2023

  94. [103]

    Are transformers universal approximators of sequence-to-sequence functions? Internation Conference on Learning Representations, 2020

    Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J Reddi, and Sanjiv Kumar. Are transformers universal approximators of sequence-to-sequence functions? Internation Conference on Learning Representations, 2020

    2020

  95. [104]

    Srs4: A stacked residual deep neural network for heave motion continuous prediction of salvage barge

    Fengrui Zhang, Jiaoyi Hou, Dayong Ning, Cheng Zhou, Gangda Liang, and Zhilei Liu. Srs4: A stacked residual deep neural network for heave motion continuous prediction of salvage barge. Available at SSRN 4938844 , 2024

    2024

  96. [105]

    A survey on visual mamba

    Hanwei Zhang, Ying Zhu, Dan Wang, Lijun Zhang, Tianxiang Chen, Ziyang Wang, and Zi Ye. A survey on visual mamba. Applied Sciences, 14(13):5683, 2024

    2024

  97. [106]

    An overview on data representation learning: From traditional feature learning to recent deep learning

    Guoqiang Zhong, Li-Na Wang, Xiao Ling, and Junyu Dong. An overview on data representation learning: From traditional feature learning to recent deep learning. The Journal of Finance and Data Science , 2(4):265–278, 2016

    2016

  98. [107]

    Universality of deep convolutional neural networks

    Ding-Xuan Zhou. Universality of deep convolutional neural networks. Applied and computational harmonic analysis , 48(2):787–794, 2020

    2020

  99. [108]

    Fed- former: Frequency enhanced decomposed transformer for long-term series forecasting

    Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fed- former: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning , pages 27268–27286. PMLR, 2022

    2022

  100. [109]

    state space models

    Lianghui Zhu, Liao Bencheng, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. International Conference on Machine Learning , 2024. 20 A Related works Sequence Models. Sequential...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.