Pith. sign in

REVIEW 3 major objections 5 minor 41 references

UnHiPPO: Uncertainty-aware Initialization for State Space Models

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that reinterpreting HiPPO as a linear stochastic control problem with noisy observations yields a new initialization, UnHiPPO, whose Kalman-filter recurrence makes state space models robust to measurement noise at no…

desk verdict A clean, well-motivated extension of HiPPO that trades exact theoretical grounding for a useful heuristic; worth reviewing, but the posterior-inference claim is only conditional on a polynomial-signal approximation the authors acknowledge. read the letter →

arxiv 2506.05065 v1 pith:TESXOB2Z submitted 2025-06-05 cs.LG stat.ML

Marten Lienen , Abdullah Saydemir , Stephan Günnemann This is my paper · ORCID
classification cs.LGstat.ML
keywords statespacemodelsHiPPOKalmanfiltermeasurementnoisesequenceclassificationLegendrepolynomialsmodelinitializationlineardynamicalsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HiPPO, the standard initialization for state space models, assumes the input signal is noise-free. The paper drops that assumption by rewriting HiPPO as a linear stochastic control problem and then swapping the noise-free control signal for noisy observations of a latent system. The resulting UnHiPPO initialization replaces the HiPPO matrices with a Kalman-filter update for the posterior mean of the latent polynomial coefficients. Because this is just a different initialization of the same linear recurrence, the model structure and runtime are unchanged while the model gains an explicit, tunable mechanism for filtering measurement noise.

What carries the argument

The machinery is the Kalman-filter posterior-mean update reduced to a single linear recurrence (Eq. 24): $m_k = (I - K_k B_H^T)\bar{A}_{R,k} m_{k-1} + K_k y_k$, defining the uncertainty-aware HiPPO matrices $\bar{A}_{U,k}$ and $\bar{B}_{U,k}$. The prior dynamics use the regularized HiPPO matrix $A_R$, obtained by solving an overdetermined system of shape-preservation conditions with a pseudo-inverse, and are discretized in closed form as the matrix exponential $\exp(\log(t_{k+1}/t_k) A_R)$. This turns the polynomial projection problem into a filtering problem while preserving the structure of the LSSL recurrence and its Krylov-kernel acceleration.

What would settle it

Run the UnHiPPO recurrence on a clean polynomial signal of degree at most $N-1$, where the Legendre projection is exact, with $\sigma^2 \to 0$: if the posterior mean does not track the known HiPPO projection coefficients, the substitution underlying the dynamics is wrong even in the noise-free case. On synthetic signals with known ground truth (e.g., Gaussian process samples with added Gaussian noise), sweep $\sigma^2$ and compare the reconstruction error of UnHiPPO against HiPPO: if no value of $\sigma^2$ yields a lower error, the claimed noise filtering does not occur.

Watch

Extended reading notes

Core claim

The central claim is that the HiPPO dynamics, which treat the observed signal $f(t)$ as a noise-free control input, can be re-derived as a continuous-discrete linear dynamical system in which $f(t)$ is a noisy observation of a latent state. Substituting the Legendre reconstruction $\hat{f}_{\le t}(t) = B_H^T c_t$ for $f(t)$ makes the dynamics data-free, and a pseudo-inverse regularization (Eq. 18) tames the extrapolation blow-up of Legendre polynomials. Filtering this system with a Kalman filter collapses to a single linear recurrence for the posterior mean, $m_k = \bar{A}_{U,k}m_{k-1} + \bar{B}_{U,k} y_k$, where $\bar{A}_{U,k}$ and $\bar{B}_{U,k}$ define the UnHiPPO initialization. The paper argues that an SSM initialized this way implicitly performs posterior inference over the projection coefficients it would have computed under noise-free HiPPO, and that this improves classification accuracy on noisy speech data both when training and test noise are matched and when they differ, with no increase in runtime.

Load-bearing premise

The derivation replaces the observed signal with its Legendre polynomial reconstruction, which is exactly correct only when the signal is a polynomial of the represented degree; for real-world noisy signals this substitution is an approximation, and the pseudo-inverse regularization used to stabilize the resulting dynamics is an additional heuristic choice.

Editorial extensions

If this is right

  • Any SSM initialized with HiPPO can switch to UnHiPPO as a drop-in replacement, gaining noise robustness with no architectural change and no runtime increase.
  • The $\sigma^2$ hyperparameter gives the user a single dial that controls how much noise is filtered, letting the model adapt to the expected noise level without retraining the backbone.
  • The closed-form matrix-exponential discretization is numerically stable on long sequences where trapezoidal and forward Euler discretizations of $A_R$ diverge, so UnHiPPO also provides a more stable way to discretize the regularized dynamics.
  • Because the posterior covariance is data-independent, the entire filter is a precomputable linear operator, so the Krylov-kernel convolution acceleration used by LSSL applies unchanged to UnLSSL.
  • On the two speech classification benchmarks, UnLSSL improves accuracy over LSSL when evaluation noise differs from training noise, and matches or exceeds it at high training noise levels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The control-to-inference rewrite is not specific to Legendre polynomials; the same substitution and filtering construction could be applied to other orthogonal-basis HiPPO variants, yielding uncertainty-aware versions of Laguerre- or Fourier-based memory.
  • Because the Kalman gain $K_k$ is data-independent, UnHiPPO is effectively a fixed linear denoiser; characterizing its frequency response (or its equivalent linear filter) could tell practitioners which noise spectra it attenuates, which the paper does not analyze.
  • The method is demonstrated on univariate speech waveforms with additive Gaussian noise; a natural test is whether it extends to multivariate or non-Gaussian measurement noise, where the scalar-observation and Gaussian assumptions would need to be relaxed.
  • The paper notes that $\sigma^2$ needs to be very large to dominate the $B_H^T P^- B_H$ term, which suggests the noise parameter encodes numerical scale as much as true measurement variance; a reparameterization (e.g., normalizing the observation equation) could make $\sigma^2$ interpretable as actual noise variance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes UnHiPPO, an alternative initialization for state space models (SSMs) that is meant to be robust to measurement noise. The authors reinterpret the HiPPO dynamics as a linear stochastic control problem, replace the noise-free control input by a noisy observation model, regularize the resulting data-free dynamics via extrapolation constraints and a pseudo-inverse, and then use a Kalman filter to derive a closed-form linear recurrence for the posterior mean. The resulting UnHiPPO matrices are used to initialize an LSSL architecture (UnLSSL). Experiments on two speech classification datasets compare UnLSSL against LSSL under various training and evaluation noise levels.

Significance. If the central claim holds, UnHiPPO is an attractive drop-in replacement for the standard HiPPO initialization: it does not change the SSM structure, does not increase runtime, and provides a tunable noise-filtering behavior through a single scalar hyperparameter. The paper is transparent about some limitations, most notably the absence of a structured parametrization for large state dimensions, and it provides a public implementation link. However, the theoretical claim that UnHiPPO performs posterior inference for the HiPPO coefficients is not fully established, because the derivation replaces the observed signal by its polynomial reconstruction at a load-bearing step. The empirical results are promising but lack error bars, so the strength of the reported improvements is difficult to assess.

major comments (3)
  1. [Section 4, Eq. (11)] The substitution f(t) = B_H^T c_t in Eq. (11) is exact only when the observed signal equals its N-term Legendre projection. For general noisy signals, the resulting data-free dynamics define a different latent process, not the evolution of the true HiPPO projection coefficients. Consequently, the Kalman filter in Eqs. (22)-(23) computes the posterior of the regularized surrogate system, and the paper's claim that UnHiPPO extends HiPPO theory to noisy data by performing posterior inference is not established as stated. I recommend adding a synthetic oracle experiment in which the ground-truth projection coefficients of a clean signal are known, comparing the UnHiPPO state against those coefficients under various noise levels, or explicitly reframing the method as a heuristic filter motivated by, rather than derived from, HiPPO.
  2. [Section 4.1, Eq. (18)] The pseudo-inverse regularization in Eq. (18) is introduced as an ad hoc modification: the overdetermined system in Eq. (17) is solved by least squares without a clear optimality criterion tied to the noise model or to the HiPPO projection objective. This step changes the dynamics even in the noise-free polynomial case, where Eq. (11) would otherwise be exact. The paper should either provide a principled justification for the two conditions in Eq. (13) and the least-squares solution, or explicitly label the regularized dynamics as a heuristic approximation and temper the theoretical claims accordingly.
  3. [Section 6, Figs. 6 and 7] The central empirical claim is that UnLSSL improves noise robustness, but Figs. 6 and 7 report accuracy differences without error bars and without stating the number of independent runs. At several noise levels the reported differences are small or negative, so without variance estimates it is unclear whether the improvements are statistically significant. Please report mean and standard deviation over multiple seeds, and preferably also the raw accuracies for both models, so that the magnitude of the effect can be judged.
minor comments (5)
  1. [Appendix C] The hyperparameter table is described as applying to the SC10 experiments, but the FSD experiments do not have a corresponding table; please report the FSD hyperparameters and the number of seeds used for all experiments.
  2. [Section 8] The limitation that no structured O(N) parametrization of the UnHiPPO matrices was found is important for scalability; consider mentioning this point earlier in the paper, since many modern SSM applications rely on structured representations.
  3. [Eqs. (20) and (22)] The index convention for the transition matrix is easy to confuse: Eq. (20) uses \bar{A}_{R,k+1} while Eq. (22) uses \bar{A}_{R,k} for the same transition from step k-1 to k. Please align the notation for readability.
  4. [Figure 5] The caption of Fig. 5 is unclear about what the labels k=500, k=100, k=20 refer to and how many times the fixed dynamics are applied; please clarify the experimental protocol used for that figure.
  5. [Equations (27), (46), and (49)] In the provided rendering, superscripts that should denote a matrix inverse appear as the characters "91"; please ensure the final PDF displays the intended exponents correctly.

Circularity Check

0 steps flagged · score 0.0 of 10

UnHiPPO's derivation is self-contained: the Kalman-filter update is rearranged, not fitted, and the empirical claims are tested against external benchmarks rather than reduced to the paper's own inputs.

full rationale

The paper derives Eq. (11) by substituting the polynomial reconstruction B_H^T c_t for f(t) in the HiPPO ODE (Eq. 4); this is an explicit modeling approximation, and the paper itself states that it is exact only when the true signal equals its own projection, so it is not a hidden circularity. The regularized dynamics in Eq. (18) are introduced as a proposal with stated goals, not imported from a self-citation or from a fitted target. Eq. (24) is obtained by algebraically combining the Kalman-filter recurrences (22)-(23), which are standard external results (Kalman 1960; Särkkä & Solin 2019); it is a rearrangement of well-known filtering equations, not a prediction forced by the data. The sigma^2 sweep in Fig. 8 tunes a hyperparameter to the evaluation noise level; this is hyperparameter selection, not a fitted parameter renamed as a prediction. No load-bearing self-citations or uniqueness theorems are invoked; references to Gu et al. are to the external HiPPO/LSSL foundations. The Limitations section (Sec. 8) states that a structured parametrization 'eluded us'; this is an acknowledged computational limitation, not a circular step. The empirical claims are evaluated on FSD and SC10 against the LSSL baseline, so the central claim is independently testable rather than equivalent to the derivation's inputs.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The method's central derivation rests on standard polynomial and Kalman-filter results, plus two ad hoc modeling choices: the substitution f(t)=B_H^T c_t and the slope-preserving regularization. The only tuned free parameter is the measurement noise variance sigma^2, which is set per dataset and is not interpretable as the actual noise level.

free parameters (1)
  • Measurement noise variance sigma^2 = 10^10 for SC10; swept in Fig 8; not estimated from data
    Controls the strength of noise filtering through the Kalman innovation covariance; the paper notes the required scale is far larger than the true noise variance, so it is a tuning knob rather than an estimated quantity.
assumptions (5)
  • standard math Legendre polynomials form an orthonormal basis for L2 on [0,t] under the shifted inner product, and the HiPPO update (Eq 4) is correct for this basis.
    Used in Sections 2-3 and Appendix A to set up the HiPPO coefficients.
  • standard math The Kalman filter provides the exact posterior mean and covariance for a linear Gaussian state space model.
    Used in Section 4.3, Eqs (22)-(23).
  • ad hoc to paper The observed signal equals its polynomial reconstruction, f(t) = B_H^T c_t, so that Eq (11) replaces the data term in the HiPPO ODE.
    This substitution is exact only for polynomial signals; for general noisy data it changes the dynamics into a different, data-free LDS. Section 4, Eq (11).
  • ad hoc to paper The regularization conditions in Eq (13), keeping the reconstruction's value and slope constant while extending the time window, define the desired extrapolation behavior.
    These are heuristic goals chosen to avoid divergence of Legendre polynomials outside their domain; they are not derived from optimal filtering. Section 4.1, Eq (13).
  • domain assumption Measurement and process noises are independent zero-mean Gaussians with variances sigma^2 and covariance Sigma.
    This is the standard linear-Gaussian assumption required for the Kalman filter; the paper fixes Sigma=I and tunes sigma^2. Eqs (7), (8), (20).

how reviews work

0 comments
Cite this review

Pith. "Pith review of UnHiPPO: Uncertainty-aware Initialization for State Space Models." pith.science (2026). https://pith.science/paper/TESXOB2Z

@misc{pith2026250605065,
  author       = {Pith},
  title        = {Pith review of: UnHiPPO: Uncertainty-aware Initialization for State Space Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TESXOB2Z}},
  note         = {Machine review of arXiv:2506.05065}
}
read the original abstract

State space models are emerging as a dominant model class for sequence problems with many relying on the HiPPO framework to initialize their dynamics. However, HiPPO fundamentally assumes data to be noise-free; an assumption often violated in practice. We extend the HiPPO theory with measurement noise and derive an uncertainty-aware initialization for state space model dynamics. In our analysis, we interpret HiPPO as a linear stochastic control problem where the data enters as a noise-free control signal. We then reformulate the problem so that the data become noisy outputs of a latent system and arrive at an alternative dynamics initialization that infers the posterior of this latent system from the data without increasing runtime. Our experiments show that our initialization improves the resistance of state-space models to noise both at training and inference time. Find our implementation at https://cs.cit.tum.de/daml/unhippo.

Figures

Figures reproduced from arXiv: 2506.05065 by the authors.

Figure 1
Figure 1. Function approximation with Legendre polynomials. The Legendre polynomials Pi : R → R, i ∈ N0 are a set of polynomials of increasing degree i orthogonal with respect to the L 2 inner product ⟨f, g⟩ = R 1 −1 f(x) g(x) dx. Together, they form a basis of the space of L 2 -integrable functions on [−1, 1] and let us represent any such function f as ˆf(x) = X D i=0 ⟨f, Pi⟩ ∥Pi∥ 2 Pi(x). (1) ˆf is the best possible approxi… view at source ↗
Figure 2
Figure 2. The Legendre polynomial ˆf≤tk diverges rapidly beyond its domain [0, tk]. If we extrapolate ˆf≤tk with Eq. (11), ˆf≤tk+1 will equal ˆf≤tk but on the extended domain [0, tk+1]. By regular￾izing the dynamics with the conditions in Eq. (13), we can get the ˆf≤tk+1 shown here, that extrapolates ˆf≤tk linearly until tk+1. This is a problem from two perspectives. First, when we model f≤t with, for example, N = 256 coeffic… view at source ↗
Figure 3
Figure 3. The uncertainty-aware HiPPO dynamics filter out the majority of the noise. The strength of the denoising is controlled by the σ 2 hyper￾parameter through its influence on the so-called innovation covariance sk in the Kalman update step in Eq. (23) [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The σ 2 hyperparameter controls the level of filtering. For the transition uncertainty, we always choose Σ = I. In our experiments, it had no appreciable effect except for instability if it was very small. 5.1. Uncertainty-aware Linear State Space Layer The LSSL (Gu et…
Figure 6
Figure 6. Figure 6: Accuracy difference between UnLSSL and LSSL in speech classification trained on a noise level ρ ′ and evaluated on a range of noise levels ρ. [−1, 1] into [−1, 1−2/k+1] and B¯ k+1 fills the freed up inter￾val [1−2/k+1, 1] with the new data. So, repeated application of …
Figure 5
Figure 5. Figure 5: Reconstruction ˆf≤tk of signal f obtained through re￾peated application of A¯k and B¯k for fixed k. Data observed at 250 equispaced locations between 0 and 10. Each application of A¯k compresses the reconstruction ˆf≤tk by k−1/k. Semantically, the dynamics A¯ k+1 of ei…
Figure 7
Figure 7. Figure 7: Accuracy difference between UnLSSL and LSSL in speech classification for models trained and evaluated on a range of noise levels ρ. 100 102 104 106 108 1010 1012 1014 σ 2 72.5 75.0 77.5 Accuracy [%] ρ = 10−5 LSSL UnLSSL [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The σ 2 parameter adapts the noise filtering to the level of noise in the data on SC10 with ρ = 0.1. Error bars denote 1 standard deviation. the data, we add independent Gaussian noise δ ∼ N (0, ρ2 ) to the training and/or test data. Noise Robustness For [PITH_FULL_IM…
Figure 9
Figure 9. Figure 9: Visualization of repeated application of HiPPO and UnHiPPO matrices with various discretization methods through the polynomial representation ˆf of c. Magnitude of the plotted curves correlates with ∥A¯i kctk ∥. C. Experiment Details We use the parameters listed in [P…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 29 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    work page

  5. [5]

    Spectral State Space Models , 2023

    Agarwal, N., Suo, D., Chen, X., and Hazan, E. Spectral State Space Models , 2023

    work page 2023

  6. [6]

    and Weber, H

    Arfken, G. and Weber, H. J. Mathematical Methods for Physicists . Elsevier Academic Press, Amsterdam, Heidelberg, 6. ed., 5. [print., international ed.] edition, 2008. ISBN 978-0-12-059876-2

    work page 2008

  7. [7]

    Astr \"o m, K. J. Introduction to Stochastic Control Theory . Number v. 70 in Mathematics in Science and Engineering. Academic Press, New York, 1970. ISBN 978-0-12-065650-9

    work page 1970

  8. [8]

    Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation , September 2014

    Cho, K., van Merrienboer , B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation , September 2014

    work page 2014

Show all 41 references
  1. [9]

    It's Raw ! Audio Generation with State-Space Models

    Goel, K., Gu, A., Donahue, C., and Re, C. It's Raw ! Audio Generation with State-Space Models . In International Conference on Machine Learning , pp.\ 7616--7633. PMLR, June 2022

    2022

  2. [10]

    What every computer scientist should know about floating-point arithmetic

    Goldberg, D. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 23 0 (1): 0 5--48, March 1991. ISSN 0360-0300. doi:10.1145/103162.103163

    1991

  3. [11]

    Granger, B. E. and P \'e rez, F. Jupyter: Thinking and Storytelling With Code and Data . Computing in Science & Engineering, 23 0 (2): 0 7--14, March 2021. ISSN 1558-366X. doi:10.1109/MCSE.2021.3059263

    2021

  4. [12]

    and Dao, T

    Gu, A. and Dao, T. Mamba: Linear-Time Sequence Modeling with Selective State Spaces , December 2023

    2023

  5. [13]

    HiPPO : Recurrent Memory with Optimal Polynomial Projections

    Gu, A., Dao, T., Ermon, S., Rudra, A., and Re, C. HiPPO : Recurrent Memory with Optimal Polynomial Projections . In Neural Information Processing Systems , 2020

    2020

  6. [14]

    Combining Recurrent , Convolutional , and Continuous-time Models with Linear State-Space Layers

    Gu, A., Johnson, I., Goel, K., Saab, K., Dao, T., Rudra, A., and R \'e , C. Combining Recurrent , Convolutional , and Continuous-time Models with Linear State-Space Layers . In Neural Information Processing Systems . arXiv, October 2021. doi:10.48550/arXiv.2110.13985

    2021 doi arXiv

  7. [15]

    Efficiently Modeling Long Sequences with Structured State Spaces

    Gu, A., Goel, K., and R \'e , C. Efficiently Modeling Long Sequences with Structured State Spaces . In International Conference on Learning Representations . arXiv, 2022 a

    2022

  8. [16]

    On the Parameterization and Initialization of Diagonal State Space Models , June 2022 b

    Gu, A., Gupta, A., Goel, K., and R \'e , C. On the Parameterization and Initialization of Diagonal State Space Models , June 2022 b

    2022

  9. [17]

    How to Train your HIPPO : State Space Models with Generalized Orthogonal Basis Projections

    Gu, A., Johnson, I., Timalsina, A., Rudra, A., and Re, C. How to Train your HIPPO : State Space Models with Generalized Orthogonal Basis Projections . In International Conference on Learning Representations , September 2022 c

    2022

  10. [18]

    Diagonal State Spaces are as Effective as Structured State Spaces

    Gupta, A., Gu, A., and Berant, J. Diagonal State Spaces are as Effective as Structured State Spaces . In Advances in Neural Information Processing Systems , October 2022

    2022

  11. [19]

    R., Millman, K

    Harris, C. R., Millman, K. J., van der Walt , S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk , M. H., Brett, M., Haldane, A., del R \'i o , J. F., Wiebe, M., Peterson, P., G \'e rar...

    2020 doi

  12. [20]

    Learning Linear Dynamical Systems via Spectral Filtering

    Hazan, E., Singh, K., and Zhang, C. Learning Linear Dynamical Systems via Spectral Filtering . In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017

    2017

  13. [21]

    and Schmidhuber, J

    Hochreiter, S. and Schmidhuber, J. Long Short-Term Memory . Neural Computation, 9: 0 1735--1780, 1997

    1997

  14. [22]

    Hunter, J. D. Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9 0 (3): 0 90--95, 2007. doi:10.1109/MCSE.2007.55

    2007 doi

  15. [23]

    Free-spoken-digit-dataset: V1.0.8

    Jackson, Z., Souza, C., Flaks, J., Pan, Y., Nicolas, H., and Thite, A. Free-spoken-digit-dataset: V1.0.8. Zenodo, August 2018

    2018

  16. [24]

    Kalman, R. E. A New Approach to Linear Filtering and Prediction Problems . Journal of Basic Engineering, 82 0 (1): 0 35--45, March 1960. ISSN 0021-9223. doi:10.1115/1.3662552

    1960 doi

  17. [25]

    and Li, Q

    Liu, F. and Li, Q. Autocorrelation Matters : Understanding the Role of Initialization Schemes for State Space Models . In International Conference on Learning Representations , 2025

    2025

  18. [26]

    Long Range Language Modeling via Gated State Spaces

    Mehta, H., Gupta, A., Cutkosky, A., and Neyshabur, B. Long Range Language Modeling via Gated State Spaces . In The Eleventh International Conference on Learning Representations , September 2022

    2022

  19. [27]

    VideoMamba : Spatio-Temporal Selective State Space Model

    Park, J., Kim, H.-S., Ko, K., Kim, M., and Kim, C. VideoMamba : Spatio-Temporal Selective State Space Model . In Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., and Varol, G. (eds.), European Conference on Computer Vision , pp.\ 1--18, Cham, 2025. Springer Na...

    2025 doi

  20. [28]

    PyTorch : An Imperative Style , High-Performance Deep Learning Library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., K \"o pf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. PyTorch : An I...

    2019

  21. [29]

    Patro, B. N. and Agneeswaran, V. S. Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling : Methods , Applications , and Challenges , April 2024 a

    2024

  22. [30]

    Patro, B. N. and Agneeswaran, V. S. SiMBA : Simplified Mamba-Based Architecture for Vision and Multivariate Time series, April 2024 b

    2024

  23. [31]

    Einops: Clear and Reliable Tensor Manipulations with Einstein-like Notation

    Rogozhnikov, A. Einops: Clear and Reliable Tensor Manipulations with Einstein-like Notation . In International Conference on Learning Representations , 2022

    2022

  24. [32]

    and Solin, A

    S \"a rkk \"a , S. and Solin, A. Applied Stochastic Differential Equations . Cambridge University Press, April 2019. ISBN 978-1-108-18673-5

    2019

  25. [33]

    Smith, J. T. H., Warrington, A., and Linderman, S. W. Simplified State Space Layers for Sequence Modeling . In International Conference on Learning Representations , 2023

    2023

  26. [34]

    Vetterli, M., Kova c evi \'c , J., and Goyal, V. K. Foundations of Signal Processing. Cambridge University Press, Cambridge, 2014. ISBN 978-1-107-03860-8

    2014

  27. [35]

    E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt , S

    Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt , S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., Carey, C. J...

    2020

  28. [36]

    Legendre Memory Units : Continuous-Time Representation in Recurrent Neural Networks

    Voelker, A., Kaji \'c , I., and Eliasmith, C. Legendre Memory Units : Continuous-Time Representation in Recurrent Neural Networks . In Neural Information Processing Systems , 2019

    2019

  29. [37]

    Speech Commands : A Dataset for Limited-Vocabulary Speech Recognition , April 2018

    Warden, P. Speech Commands : A Dataset for Limited-Vocabulary Speech Recognition , April 2018

    2018

  30. [38]

    Hydra - A framework for elegantly configuring complex applications

    Yadan, O. Hydra - A framework for elegantly configuring complex applications. Github, 2019

    2019

  31. [39]

    W., and Erichson, N

    Yu, A., Nigmetov, A., Morozov, D., Mahoney, M. W., and Erichson, N. B. Robustifying State-space Models for Long Sequences via Approximate Diagonalization . In International Conference on Learning Representations . arXiv, October 2023. doi:10.48550/arXiv.2310.01698

    2023 doi arXiv

  32. [40]

    W., and Erichson, N

    Yu, A., Mahoney, M. W., and Erichson, N. B. HOPE for a Robust Parameterization of Long-memory State Space Models . In International Conference on Learning Representations . arXiv, 2025. doi:10.48550/arXiv.2405.13975

    2025 doi arXiv

  33. [41]

    Kalman- SSM : Modeling long-term time series with kalman filter structured state spaces

    Zhou, Z., Guo, X., Jie, Y., and Xia, C.-M. Kalman- SSM : Modeling long-term time series with kalman filter structured state spaces. IEEE Signal Processing Letters, PP: 0 1--5, January 2024. doi:10.1109/LSP.2024.3457862

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.