Pith. sign in

REVIEW 3 major objections 5 minor 160 references

Deep Learning-based Approaches for State Space Models: A Selective Review

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A selective review argues that deep state-space models—from neural Kalman filters through latent ODEs/SDEs to S4 and Mamba—fit one unified template.

desk verdict Useful selective review of deep SSMs, but its VAE-centric framing contradicts its own surveyed works, and the S4 derivation has a real gap. read the letter →

arxiv 2412.11211 v1 pith:UABN6P4H submitted 2024-12-15 stat.ML cs.LGstat.OT

Jiahe Lin , George Michailidis This is my paper · ORCID
classification stat.MLcs.LGstat.OT MSC 62M1062M2068T0760H10
keywords statespacemodelsvariationalautoencodersneuralODEsSDEsstructuredsequencemodelingmixedfrequencytimeseriesirregularly-spaced
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that recent deep-learning state-space models are variations on one template rather than separate inventions. The template is a two-equation structure: a latent state process that evolves in discrete or continuous time, and an observation map that links states to data, learned through a variational autoencoder that encodes an approximate posterior and decodes reconstructions. The same equations reappear as standalone neural-network layers for efficient long-sequence modeling, where the technical work is about making the state-transition matrix structured so recurrence, convolution, and parallel scan all become cheap. A sympathetic reader would care because the synthesis gives a common vocabulary for comparing neural Kalman filters, latent neural ODEs/SDEs, and modern sequence layers such as S4, S5, and Mamba.

What carries the argument

The load-bearing object is the two-equation state-space template, $z_t = f(z_{t-1}, u_t)$ and $x_t = g(z_t, \epsilon_t)$ (or their continuous-time counterparts), together with the learning pipeline built on the variational evidence lower bound (ELBO). The ELBO splits into reconstruction terms and KL-divergence terms between the encoder and the prior or state-transition distributions; this decomposition is what lets an encoder network stand in for filtering and smoothing. In the architectural branch, the key objects are the linear state-space layer with state matrix $A$, input map $B$, output map $C$, and timescale $\Delta$, the structured HiPPO initialization that gives the state long memory, and the diagonal-plus-low-rank parameterization, generating-function or parallel-scan algorithms, and input-dependent timescale selection that turn exponential cost into near-linear cost.

What would settle it

One concrete test: assemble the population of neural SSM papers since 2021 and record the training objective. If a sizable share of prominent models are trained without any ELBO—by direct likelihood, Kalman-style marginalization, or simulation-based inference—the 'de facto standard' claim fails; if a major line fits neither the generative latent-model branch nor the standalone-layer branch, the unified taxonomy fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that a unified perspective exists for discrete-time deep state-space models and continuous-time latent neural ODE/SDE models. All of them share the same skeleton of a state equation plus an observation equation, and they differ mainly in whether time is discrete or continuous and in how the posterior over latent states is approximated. The review further asserts that VAE-based training—an encoder that approximates the posterior and a decoder that reconstructs observations from sampled states—has become the de facto standard for deep generative SSMs, replacing explicit filtering and smoothing recursions where those are intractable. On the architectural side, it treats linear state-space layers and their structured successors as the same mathematical object used as a building block for efficient sequence modeling, with HiPPO-based initialization, diagonal-plus-low-rank parameterization, and selective input-dependent timescales as successive refinements.

Load-bearing premise

The synthesis rests on the selection of representative papers: if the chosen examples do not faithfully represent the broader literature, the claimed unity and the 'de facto' status of VAE training could be selection artifacts.

Editorial extensions

If this is right

  • Classical filtering and smoothing and deep VAE training are two routes to the same posterior-over-latent-states problem, so ideas like smoothing can inform encoder design in deep SSMs, and vice versa.
  • Latent neural ODEs and SDEs occupy a continuum with discrete-time deep SSMs: discrete models become ODE-like as step sizes shrink, and SDEs add a diffusion term, with the different encoder and decoder layouts forced by how the posterior factorizes over time.
  • In the architectural line, the same linear state-space equations support three equivalent views—recurrence, convolution, and continuous-time dynamics—so S4, S5, and Mamba can be trained in parallel and generated from sequentially by switching representations.
  • For mixed-frequency and irregularly-spaced data, the SSM formulation has a structural advantage because the latent state lives on a common or continuous time axis, letting one model produce forecasts, nowcasts, imputations, and extrapolations without retraining per vintage.
  • If the unified perspective holds, researchers can transfer algorithmic advances across branches, for example using structured-matrix tricks from the architectural line to speed up inference in generative latent SSMs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same state-plus-observation template could serve as a benchmark testbed: instantiate the same encoder and observation map while swapping discrete, ODE, SDE, and selective state dynamics, isolating what each state-equation choice contributes to accuracy and efficiency.
  • The paper's two-branch split into generative latent model versus standalone layer is probably already blurring, since architectural SSMs are being connected back to attention and to input-dependent latent dynamics; a future unified account may treat Transformers and SSMs as endpoints of the same continuum.
  • A testable extension of the 'VAE as de facto standard' claim is to count training objectives in recent neural-SSM publications; if direct likelihood estimation or simulation-based inference occupies a nontrivial share, the standard would need qualification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper is a selective review of deep learning-based approaches to state space models (SSMs). It aims to present a unified perspective that covers discrete-time deep SSMs, continuous-time latent neural ODE/SDEs, and recent SSM-derived sequence-modeling architectures (LSSL, S4, S5, Mamba). The review begins with classical maximum-likelihood-based learning, including filtering/smoothing and the EM algorithm, then describes the variational autoencoder (VAE) pipeline as the dominant learning framework for latent-variable neural SSMs. It subsequently surveys representative deep SSM papers organized by discrete-time, latent ODE, and latent SDE formulations, discusses additional related approaches, and closes with applications to mixed-frequency and irregularly-spaced time series. Appendices provide background on Kalman filtering, nonlinear filters, and SDE-based SSMs.

Significance. If the synthesis were accurate, this review would be a useful bridge between two largely separate literatures: the statistical latent-variable SSM tradition and the modern deep learning sequence-modeling tradition. The paper covers a broad range of material in a clear, well-organized manner, and its derivations of the ELBO, filtering recursions, and the LSSL/S4 machinery are mostly standard and readable. The review also has the merit of making the connections between classical filtering and modern encoder-decoder pipelines explicit. However, the central claim of a unified perspective is weakened by an internal inconsistency in the taxonomy: the paper asserts that VAE-based training is the de facto standard for deep SSMs, yet it reviews several non-VAE neural SSMs without reconciling them with that claim. A corrected and more carefully qualified taxonomy would make this a valuable review for researchers entering the field.

major comments (3)
  1. [Section 2.3 and Section 6] The assertion that VAE-based training is 'the de facto approach' (Section 2.3) and 'the dominant learning paradigm' (Section 6) is contradicted by models reviewed elsewhere in the same paper. Section 3.3.1 explicitly states that Jia and Benson (2019) 'does not rely on a VAE-based training pipeline' and instead uses the adjoint method; Deng et al. (2021) is presented without a VAE objective; Masti and Bemporad (2021) in Section 3.4 uses an autoencoder with reconstruction plus state-consistency losses, not an ELBO; and Section 4 models (S4, S5, Mamba) are deterministic SSM modules trained with standard sequence objectives. Since the taxonomy in Section 2.3 is organized around the 'partially DL vs fully DL with VAE' distinction, these non-VAE models fall outside its organizing principle, making the claimed unified perspective overstate VAE's role. Please revise the claim to a more qualified statement, e.g., 'VAE is the dominant paradigm in the latent generative strand,' or provide a tripartite taxonomy that accommodates the non-VAE fully DL models.
  2. [Section 4.2, Eq. (4.5) and footnote 11] The derivation of the generating function identity is mathematically incorrect. The finite geometric sum formula gives sum_{ell=0}^{L-1} C \bar{A}^ell \bar{B} \phi^ell = C(I - \bar{A}^L \phi^L)(I - \bar{A}\phi)^{-1}\bar{B}. The last equality in footnote 11, which drops the factor (I - \bar{A}^L \phi^L), holds only if \bar{A}^L = I, not merely because \phi^L = 1. Since \bar{A} is a general structured matrix, the asserted identity in (4.5) does not follow. This is not a minor typographical issue: it misrepresents the S4 technique, which is one of the central topics of Section 4. Please correct the presentation, for instance by stating the finite-sum identity with the correction term and explaining how the S4 algorithm handles it, or by deferring to the original derivation in Gu et al. (2022).
  3. [Section 3.4 and Section 2.3] The classification of Masti and Bemporad (2021) as falling under the 'fully DL' category is inconsistent with the definition in Section 2.3, which states that fully DL approaches learn parameters with a VAE-based pipeline. The loss described in Section 3.4 is a sum of a reconstruction error and a state-consistency discrepancy, with no ELBO or variational posterior. This is an internal inconsistency, not merely an omission: the paper's own taxonomy assigns the model to a category that its definition does not fit. Please either reclassify the method or amend the definition of the fully DL category to include non-VAE training objectives such as reconstruction-plus-consistency losses.
minor comments (5)
  1. [Section 5.2] The sentence 'Detailed explanations if these two approaches can be found in the respective references' contains a typo: 'if' should be 'of'.
  2. [Section 3.3, Eq. (3.10)] The notation '(\Delta t) Id' is ambiguous; it should be '(\Delta t) I_d' to indicate the d-dimensional identity matrix.
  3. [Section 2.3, Eq. (2.10)] The KL divergence terms contain a duplicated vertical bar, 'KL(q(.) || p(.))' is written with two pairs of parallel bars; one pair should be removed for clarity.
  4. [Section 4.2, footnote 11] The phrase '~C collects the constant term' is vague. It would be clearer to state that the derivation incorrectly treats the truncated sum as if it were an infinite sum; the intended constant term appears after using \phi^L = 1 in a corrected finite-sum identity.
  5. [Section 5.2] The description of the forecasting variant for Rubanova et al. (2019) partitions the observed time points into two halves, but it is not specified whether the split is by time index or by sorted timestamps; please clarify.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the review's synthesis is built on external literature and standard derivations, with the authors' own citations confined to non-load-bearing remarks.

full rationale

The paper is a selective review, not a derivation of new results. Its central organizational claims—the VAE/ELBO pipeline for deep SSMs and the discrete/continuous latent process taxonomy—are drawn from the cited external literature (Kingma and Welling 2013; Krishnan et al. 2015, 2017; Rubanova et al. 2019; Hasan et al. 2021; Gu et al. 2021, 2022; Smith et al. 2023; Gu and Dao 2023), not from the authors' own prior work. The authors' self-citations (Lin and Michailidis 2020a in a factor-model analogy, 2020b in a remark on serially correlated observation errors, and 2023 as a pointer to a MIDAS/VAR comparison) are illustrative asides and never carry the argument. The equations in the review—the likelihood recursion (2.4)-(2.9), the ELBO (2.10), the neural ODE/SDE formulations (3.5)-(3.10), and the LSSL/S4/S5/Mamba state-space layers (4.1)-(4.6)—are standard textbook derivations or faithful restatements of the referenced papers, with no fitted parameter repackaged as a prediction and no uniqueness claim imported from the authors' own prior theorems. Although the assertion that VAE is the 'de facto' training pipeline (Sections 2.3 and 6) is a generalization that sits somewhat uneasily with non-VAE examples such as Jia and Benson (2019), that is a question of internal consistency or coverage, not circularity: the review's organizing perspective is not defined into existence by its own conclusion, and the non-VAE models are explicitly reviewed rather than hidden. No circular step can be exhibited by matching equations or by a self-citation chain, so the correct finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The review introduces no free parameters or invented entities; it is a synthesis of existing methods. The axioms listed are standard domain assumptions inherited from the cited literature, needed for the classical and continuous-time results the review summarizes.

assumptions (3)
  • domain assumption State-space models are identifiable: theta != theta' implies p_theta(x_1:T) differs from p_theta'(x_1:T) on a set of positive measure.
    Section 2.1, footnote 1. The MLE formulation in the review depends on identifiability for parameter estimation.
  • domain assumption Nonlinear SDEs in Appendix C have unique pathwise solutions when drift and diffusion are globally Lipschitz and the observation function satisfies linear growth.
    Section 3.3 and Appendix C (C.1a)-(C.1b). The review relies on these well-posedness conditions to state the latent SDE models.
  • standard math Kalman filter and smoother recursions for linear Gaussian SSMs are correct.
    Appendix A. The review uses these recursions as a baseline for comparison with deep learning methods.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Learning-based Approaches for State Space Models: A Selective Review." pith.science (2026). https://pith.science/paper/UABN6P4H

@misc{pith2026241211211,
  author       = {Pith},
  title        = {Pith review of: Deep Learning-based Approaches for State Space Models: A Selective Review},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UABN6P4H}},
  note         = {Machine review of arXiv:2412.11211}
}
read the original abstract

State-space models (SSMs) offer a powerful framework for dynamical system analysis, wherein the temporal dynamics of the system are assumed to be captured through the evolution of the latent states, which govern the values of the observations. This paper provides a selective review of recent advancements in deep neural network-based approaches for SSMs, and presents a unified perspective for discrete time deep state space models and continuous time ones such as latent neural Ordinary Differential and Stochastic Differential Equations. It starts with an overview of the classical maximum likelihood based approach for learning SSMs, reviews variational autoencoder as a general learning pipeline for neural network-based approaches in the presence of latent variables, and discusses in detail representative deep learning models that fall under the SSM framework. Very recent developments, where SSMs are used as standalone architectural modules for improving efficiency in sequence modeling, are also examined. Finally, examples involving mixed frequency and irregularly-spaced time series data are presented to demonstrate the advantage of SSMs in these settings.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

160 extracted references · 69 canonical work pages

  1. [1]

    A t-Sahalia, Y. (2004). Disentangling diffusion from jumps. Journal of Financial Economics\/ 74\/ (3), 487--528

    2004

  2. [2]

    Anderson, B. D. and J. B. Moore (2005). Optimal filtering . Courier Corporation

    2005

  3. [3]

    Izzo, and G

    Andreini, P., C. Izzo, and G. Ricco (2020). Deep dynamic factor models. arXiv preprint arXiv:2007.11887\/

    arXiv 2020

  4. [4]

    Doucet, S

    Andrieu, C., A. Doucet, S. S. Singh, and V. B. Tadic (2004). Particle methods for change detection, system identification, and control. Proceedings of the IEEE\/ 92\/ (3), 423--438

    2004

  5. [5]

    Unosson, and Y

    Ankargren, S., M. Unosson, and Y. Yang (2020). A flexible mixed-frequency vector autoregression with a steady-state prior. Journal of Time Series Econometrics\/ 12\/ (2)

    2020

  6. [6]

    Aoki, M. (2013). State space modeling of time series . Springer Science & Business Media

    2013

  7. [7]

    Archer, E., I. M. Park, L. Buesing, J. Cunningham, and L. Paninski (2016). Black box variational inference for state space models. In International Conference on Learning Representations Workshop

    2016

  8. [8]

    Tornero, and M

    Armesto, L., J. Tornero, and M. Vincze (2008). On multi-rate fusion for non-linear sampled-data systems: Application to a 6D tracking system. Robotics and Autonomous Systems\/ 56\/ (8), 706--715

    2008

Show all 160 references
  1. [9]

    Li, and L

    Bai, J., K. Li, and L. Lu (2016). Estimation and inference of FAVAR models. Journal of Business & Economic Statistics\/ 34\/ (4), 620--641

    2016

  2. [10]

    Bain, A. and D. Crisan (2009). Fundamentals of Stochastic Filtering , Volume 3. Springer

    2009

  3. [11]

    Bar-Shalom, Y., X. R. Li, and T. Kirubarajan (2004). Estimation with applications to tracking and navigation: theory algorithms and software . John Wiley & Sons

    2004

  4. [12]

    Bass, R. F. (2004). Stochastic differential equations with jumps . Probability Surveys\/ 1 , 1 -- 19

    2004

  5. [13]

    Bell, B. M. and F. W. Cathey (1993). The iterated K alman filter update as a G auss- N ewton method. IEEE Transactions on Automatic Control\/ 38\/ (2), 294--297

    1993

  6. [14]

    Bernanke, B. S., J. Boivin, and P. Eliasz (2005). Measuring the effects of monetary policy: a factor-augmented vector autoregressive ( FAVAR ) approach. The Quarterly Journal of Economics\/ 120\/ (1), 387--422

    2005

  7. [15]

    Papaspiliopoulos, G

    Beskos, A., O. Papaspiliopoulos, G. O. Roberts, and P. Fearnhead (2006). Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion). Journal of the Royal Statistical Society Series B: Statistical Methodology\/ ...

    2006

  8. [16]

    Beskos, A. and G. Roberts (2005). Exact simulation of diffusions. The Annals of Applied Probability\/ 15\/ (4), 2422--2444

    2005

  9. [17]

    Sommer, S

    Bilo s , M., J. Sommer, S. S. Rangapuram, T. Januschowski, and S. G \"u nnemann (2021). Neural flows: Efficient alternative to neural ODE s. Advances in Neural Information Processing Systems\/ 34 , 21325--21337

    2021

  10. [18]

    Blanchet, J. and F. Zhang (2020). Exact simulation for multivariate it \^o diffusions. Advances in Applied Probability\/ 52\/ (4), 1003--1034

    2020

  11. [19]

    Blelloch, G. E. (1990). Prefix sums and their applications

    1990

  12. [20]

    Doucet, and S

    Briers, M., A. Doucet, and S. Maskell (2010). Smoothing algorithms for state space models. Annals of the Institute of Statistical Mathematics\/ 62 , 61--89

    2010

  13. [21]

    Brown, T., B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Ch...

    2020

  14. [22]

    Carter, C. K. and R. Kohn (1994). On G ibbs sampling for state space models. Biometrika\/ 81\/ (3), 541--553

    1994

  15. [23]

    Carter, C. K. and R. Kohn (1996). Markov chain M onte C arlo in conditionally G aussian state space models. Biometrika\/ 83\/ (3), 589--601

    1996

  16. [24]

    Chang, C.-B. and M. Athans (1978). State estimation for discrete systems with switching parameters. IEEE Transactions on Aerospace and Electronic Systems\/ (3), 418--425

    1978

  17. [25]

    Chang, J. and S. X. Chen (2011). On the approximate maximum likelihood estimation for diffusion processes. Annals of statistics\/ 39\/ (6), 2820--2851

    2011

  18. [26]

    Purushotham, K

    Che, Z., S. Purushotham, K. Cho, D. Sontag, and Y. Liu (2018). Recurrent neural networks for multivariate time series with missing values. Scientific Reports\/ 8\/ (1), 6085

    2018

  19. [27]

    Purushotham, G

    Che, Z., S. Purushotham, G. Li, B. Jiang, and Y. Liu (2018). Hierarchical deep generative models for multi-rate multivariate time series. In International Conference on Machine Learning , pp.\ 784--793. PMLR

    2018

  20. [28]

    Chen, R. T., Y. Rubanova, J. Bettencourt, and D. K. Duvenaud (2018). Neural ordinary differential equations. Advances in Neural Information Processing Systems\/ 31

    2018

  21. [29]

    Kastner, L

    Chung, J., K. Kastner, L. Dinh, K. Goel, A. C. Courville, and Y. Bengio (2015). A recurrent latent variable model for sequential data. Advances in Neural Information Processing Systems\/ 28

    2015

  22. [30]

    Cirone, N. M., A. Orvieto, B. Walker, C. Salvi, and T. Lyons (2024). Theoretical foundations of deep selective state-space models. arXiv preprint arXiv:2402.19047\/

    2024 arXiv

  23. [31]

    Herbei, G

    Craigmile, P., R. Herbei, G. Liu, and G. Schneider (2023). Statistical inference for stochastic differential equations. Wiley Interdisciplinary Reviews: Computational Statistics\/ 15\/ (2), e1585

    2023

  24. [32]

    Dai, Z., Z. Yang, Y. Yang, J. Carbonell, Q. Le, and R. Salakhutdinov (2019). Transformer- XL : Attentive language models beyond a fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Association for Computational Li...

    2019

  25. [33]

    Dao, T. and A. Gu (2024). Transformers are SSM s: Generalized models and efficient algorithms through structured state space duality

    2024

  26. [34]

    Davis, M. H. and S. I. Marcus (1981). An introduction to nonlinear filtering. In Stochastic Systems: The Mathematics of Filtering and Identification and Applications: Proceedings of the NATO Advanced Study Institute held at Les Arcs, Savoie, France, June 22--July 5, 1980 , pp....

    1981

  27. [35]

    De Brouwer, E., J. Simm, A. Arany, and Y. Moreau (2019). GRU - ODE - B ayes: Continuous modeling of sporadically-observed time series. Advances in Neural Information Processing Systems\/ 32

    2019

  28. [36]

    Deng, R., M. A. Brubaker, G. Mori, and A. Lehrmann (2021). Continuous latent process flows. Advances in Neural Information Processing Systems\/ 34 , 5162--5173

    2021

  29. [37]

    Chang, M

    Deng, R., B. Chang, M. A. Brubaker, G. Mori, and A. Lehrmann (2020). Modeling continuous stochastic processes with dynamic normalizing flows. Advances in Neural Information Processing Systems\/ 33 , 7805--7815

    2020

  30. [38]

    Chang, K

    Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805\/

    2018 arXiv

  31. [39]

    Doucet, A., A. M. Johansen, et al. (2009). A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of Nonlinear Filtering\/ 12\/ (656-704), 3

    2009

  32. [40]

    Giannone, and L

    Doz, C., D. Giannone, and L. Reichlin (2011). A two-step estimator for large approximate dynamic factor models based on K alman filtering. Journal of Econometrics\/ 164\/ (1), 188--205

    2011

  33. [41]

    Bohner, J

    Duncker, L., G. Bohner, J. Boussard, and M. Sahani (2019). Learning interpretable continuous-time models of latent stochastic dynamical systems. In International Conference on Machine Learning , pp.\ 1726--1734. PMLR

    2019

  34. [42]

    Durbin, J. and S. J. Koopman (2000). Time series analysis of non- G aussian observations based on state space models from both classical and B ayesian perspectives. Journal of the Royal Statistical Society Series B: Statistical Methodology\/ 62\/ (1), 3--56

    2000

  35. [43]

    Durbin, J. and S. J. Koopman (2002). A simple and efficient simulation smoother for state space time series analysis. Biometrika\/ 89\/ (3), 603--616

    2002

  36. [44]

    Durbin, J. and S. J. Koopman (2012). Time Series Analysis by State Space Methods , Volume 38. OUP Oxford

    2012

  37. [45]

    Chib, and N

    Elerian, O., S. Chib, and N. Shephard (2001). Likelihood inference for discretely observed nonlinear diffusions. Econometrica\/ 69\/ (4), 959--993

    2001

  38. [46]

    Eraker, B. (2001). MCMC analysis of diffusion models with application to finance. Journal of Business & Economic Statistics\/ 19\/ (2), 177--191

    2001

  39. [47]

    Kamronn, U

    Fraccaro, M., S. Kamronn, U. Paquet, and O. Winther (2017). A disentangled recognition and nonlinear dynamics model for unsupervised learning. Advances in Neural Information Processing Systems\/ 30

    2017

  40. [48]

    Friedland, B. (2005). Control system design: an introduction to state-space methods . Courier Corporation

    2005

  41. [49]

    Fr \"u hwirth-Schnatter, S. (2001). Fully B ayesian analysis of switching G aussian state space models. Annals of the Institute of Statistical Mathematics\/ 53\/ (1), 31--49

    2001

  42. [50]

    Gallant, R. and J. R. Long (1997). Estimating stochastic daerential equations efficiently by minimum chi-squared. Biometrika\/ 84\/ (1), 125--141

    1997

  43. [51]

    Gao, Y., E. W. Archer, L. Paninski, and J. P. Cunningham (2016). Linear dynamical neural population models through nonlinear embeddings. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Advances in Neural Information Processing Systems , Volume 29. Curran A...

    2016

  44. [52]

    Koop, and A

    Gefang, D., G. Koop, and A. Poon (2020). Computationally efficient inference in large B ayesian mixed frequency VAR s. Economics Letters\/ 191 , 109120

    2020

  45. [53]

    Gelb, A. (1974). Applied Optimal Estimation . MIT Press

    1974

  46. [54]

    Ghahramani, Z. and G. E. Hinton (2000). Variational learning for switching state-space models. Neural Computation\/ 12\/ (4), 831--864

    2000

  47. [55]

    Sinko, and R

    Ghysels, E., A. Sinko, and R. Valkanov (2007). MIDAS regressions: Further results and new directions. Econometric Reviews\/ 26\/ (1), 53--90

    2007

  48. [56]

    Reichlin, and D

    Giannone, D., L. Reichlin, and D. Small (2008). Nowcasting: The real-time informational content of macroeconomic data. Journal of Monetary Economics\/ 55\/ (4), 665--676

    2008

  49. [57]

    Leglaive, X

    Girin, L., S. Leglaive, X. Bie, J. Diard, T. Hueber, and X. Alameda-Pineda (2021). Dynamical variational autoencoders: A comprehensive review. Foundations and Trends® in Machine Learning\/ 15\/ (1-2), 1--175

    2021

  50. [58]

    Gobet, E. (2002). LAN property for ergodic diffusions with discrete observations. In Annales de l'Institut Henri Poincare (B) Probability and Statistics , Volume 38, pp.\ 711--737. Elsevier

    2002

  51. [59]

    Monfort, and E

    Gourieroux, C., A. Monfort, and E. Renault (1993). Indirect inference. Journal of Applied Econometrics\/ 8\/ (S1), S85--S118

    1993

  52. [60]

    Gu, A. and T. Dao (2023). Mamba: Linear-time sequence modeling with selective state spaces

    2023

  53. [61]

    Gu, A., T. Dao, S. Ermon, A. Rudra, and C. R \'e (2020). HiPPO : Recurrent memory with optimal polynomial projections. Advances in Neural Information Processing Systems\/ 33 , 1474--1487

    2020

  54. [62]

    Gu, A., K. Goel, A. Gupta, and C. R \'e (2022). On the parameterization and initialization of diagonal state space models. Advances in Neural Information Processing Systems\/ 35 , 35971--35983

    2022

  55. [63]

    Goel, and C

    Gu, A., K. Goel, and C. Re (2022). Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations

    2022

  56. [64]

    Johnson, K

    Gu, A., I. Johnson, K. Goel, K. Saab, T. Dao, A. Rudra, and C. R \'e (2021). Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in neural information processing systems\/ 34 , 572--585

    2021

  57. [65]

    Gu, and J

    Gupta, A., A. Gu, and J. Berant (2022). Diagonal state spaces are as effective as structured state spaces. Advances in Neural Information Processing Systems\/ 35 , 22982--22994

    2022

  58. [66]

    Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica\/ , 357--384

    1989

  59. [67]

    Hamilton, J. D. (2020). Time Series Analysis . Princeton University Press

    2020

  60. [68]

    Hannan, E. J. and M. Deistler (2012). The statistical theory of linear systems . SIAM

    2012

  61. [69]

    Harrison, P. J. and C. F. Stevens (1976). Bayesian forecasting. Journal of the Royal Statistical Society Series B: Statistical Methodology\/ 38\/ (3), 205--228

    1976

  62. [70]

    Harvey, A. C. (1990). Forecasting, structural time series models and the K alman filter

    1990

  63. [71]

    Hasan, A., J. M. Pereira, S. Farsiu, and V. Tarokh (2021). Identifying latent stochastic differential equations. IEEE Transactions on Signal Processing\/ 70 , 89--104

    2021

  64. [72]

    Zhang, S

    He, K., X. Zhang, S. Ren, and J. Sun (2016). Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp.\ 770--778

    2016

  65. [73]

    Ikeda, N. and S. Watanabe (2014). Stochastic differential equations and diffusion processes . Elsevier

    2014

  66. [74]

    Ito, K. and K. Xiong (2000). Gaussian filters for nonlinear filtering problems. IEEE Transactions on Automatic Control\/ 45\/ (5), 910--927

    2000

  67. [75]

    Jia, J. and A. R. Benson (2019). Neural jump stochastic differential equations. Advances in Neural Information Processing Systems\/ 32

    2019

  68. [76]

    Julier, S. J. and J. K. Uhlmann (2004). Unscented filtering and nonlinear estimation. Proceedings of the IEEE\/ 92\/ (3), 401--422

    2004

  69. [77]

    Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering\/ 82 , 32--45

    1960

  70. [78]

    Kalman, R. E. and R. S. Bucy (1961). New results in linear filtering and prediction theory. Journal of Basic Engineering\/ 83 , 95--108

    1961

  71. [79]

    Doucet, S

    Kantas, N., A. Doucet, S. S. Singh, J. Maciejowski, and N. Chopin (2015). On particle methods for parameter estimation in state-space models. Statistical Science\/ 30\/ (3), 328 -- 351

    2015

  72. [80]

    Soelch, J

    Karl, M., M. Soelch, J. Bayer, and P. van der Smagt (2017). Deep variational B ayes filters: Unsupervised learning of state space models from raw data. In International Conference on Learning Representations

    2017

  73. [81]

    Katharopoulos, A., A. Vyas, N. Pappas, and F. Fleuret (2020). Transformers are RNN s: Fast autoregressive transformers with linear attention. In International Conference on Machine Learning , pp.\ 5156--5165. PMLR

    2020

  74. [82]

    Kim, C.-J. (1994). Dynamic linear models with M arkov-switching. Journal of Econometrics\/ 60\/ (1-2), 1--22

    1994

  75. [83]

    Kim, C.-J. and C. R. Nelson (1998). Business cycle turning points, a new coincident index, and tests of duration dependence based on a dynamic factor model with regime switching. Review of Economics and Statistics\/ 80\/ (2), 188--201

    1998

  76. [84]

    Kim, C.-J. and C. R. Nelson (1999). State-space models with regime switching: classical and G ibbs-sampling approaches with applications. MIT Press Books\/ 1

    1999

  77. [85]

    Kingma, D. P. and M. Welling (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114\/

    2013 arXiv

  78. [86]

    Kaiser, and A

    Kitaev, N., . Kaiser, and A. Levskaya (2020). Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451\/

    2020 arXiv

  79. [87]

    Kitagawa, G. (1987). Non-gaussian state—space modeling of nonstationary time series. Journal of the American statistical association\/ 82\/ (400), 1032--1041

    1987

  80. [88]

    Kitagawa, G. (1988). Numerical approach to non- G aussian smoothing and its applications. In 20th Interface Symposium Computer Science and Statistics , pp.\ 379--388

    1988

  81. [89]

    Kitagawa, G. and W. Gersch (2012). Smoothness Priors Analysis of Time Series , Volume 116. Springer Science & Business Media

    2012

  82. [90]

    Kloeden, P. E. and E. Platen (1992). Stochastic Differential Equations . Springer

    1992

  83. [91]

    Kobyzev, I., S. J. Prince, and M. A. Brubaker (2020). Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence\/ 43\/ (11), 3964--3979

    2020

  84. [92]

    Shalit, and D

    Krishnan, R., U. Shalit, and D. Sontag (2015). Deep kalman filters. In Neurips Workshop on the Advances in Approximate Bayesian Inference

    2015

  85. [93]

    Shalit, and D

    Krishnan, R., U. Shalit, and D. Sontag (2017). Structured inference networks for nonlinear state space models. In Proceedings of the AAAI Conference on Artificial Intelligence , Volume 31

    2017

  86. [94]

    Kumar, P. R. and P. Varaiya (2015). Stochastic systems: Estimation, identification, and adaptive control . SIAM

    2015

  87. [95]

    LeGland, F. (2005). Splitting-up approximation for SPDE s and SDE s with application to nonlinear filtering. In Stochastic Partial Differential Equations and Their Applications: Proceedings of IFIP WG 7/1 International Conference University of North Carolina at Charlotte, NC J...

    2005

  88. [96]

    Li, X., T.-K. L. Wong, R. T. Chen, and D. Duvenaud (2020). Scalable gradients for stochastic differential equations. In International Conference on Artificial Intelligence and Statistics , pp.\ 3870--3882. PMLR

    2020

  89. [97]

    Li, Y. and S. Mandt (2018). Disentangled sequential autoencoder. In International Conference on Machine Learning , pp.\ 5670--5679. PMLR

    2018

  90. [98]

    Lin, J. and G. Michailidis (2020a). Regularized estimation of high-dimensional factor-augmented vector autoregressive ( FAVAR ) models. Journal of Machine Learning Research\/ 21\/ (117), 1--51

    2020

  91. [99]

    Lin, J. and G. Michailidis (2020b). System identification of high-dimensional linear dynamical systems with serially correlated output noise components. IEEE Transactions on Signal Processing\/ 68 , 5573--5587

    2020

  92. [100]

    Lin, J. and G. Michailidis (2023). A multi-task encoder-dual-decoder framework for mixed frequency data prediction. International Journal of Forecasting\/

    2023

  93. [101]

    Lipton, Z. C., D. Kale, and R. Wetzel (2016). Directly modeling missing data in sequences with rnns: Improved classification of clinical time series. In Machine learning for healthcare conference , pp.\ 253--270. PMLR

    2016

  94. [102]

    Liu, J. S. and R. Chen (1998). Sequential monte carlo methods for dynamic systems. Journal of the American statistical association\/ 93\/ (443), 1032--1044

    1998

  95. [103]

    Liu, X., T. Xiao, S. Si, Q. Cao, S. Kumar, and C.-J. Hsieh (2020, June). How does noise help robustness? explanation and exploration under the neural sde framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  96. [104]

    Mikulevicius, and B

    Lototsky, S., R. Mikulevicius, and B. L. Rozovskii (1997). Nonlinear filtering revisited: a spectral approach. SIAM Journal on Control and Optimization\/ 35\/ (2), 435--461

    1997

  97. [105]

    L \"u tkepohl, H. (2005). New introduction to multiple time series analysis . Springer Science & Business Media

    2005

  98. [106]

    Marlin, B. M., D. C. Kale, R. G. Khemani, and R. C. Wetzel (2012). Unsupervised pattern discovery in electronic health care data using probabilistic clustering models. In Proceedings of the 2nd ACM SIGHIT international health informatics symposium , pp.\ 389--398

    2012

  99. [107]

    Maruyama, G. (1955). Continuous M arkov processes and stochastic equations. Rendiconti del Circolo Matematico di Palermo\/ 4 , 48--90

    1955

  100. [108]

    Masti, D. and A. Bemporad (2021). Learning nonlinear state-space models using autoencoders. Automatica\/ 129 , 109666

    2021

  101. [109]

    McCracken, M. W., M. Owyang, and T. Sekhposyan (2015). Real-time forecasting with a large mixed frequency B ayesian VAR . FRB St. Louis Working Paper\/ (2015-30)

    2015

  102. [110]

    Fearnhead, and L

    Nemeth, C., P. Fearnhead, and L. Mihaylova (2016). Particle approximations of the score and observed information matrix for parameter estimation in state--space models with linear computational cost. Journal of Computational and Graphical Statistics\/ 25\/ (4), 1138--1157

    2016

  103. [111]

    Ocone, D. and E. Pardoux (2006). A L ie algebraic criterion for non-existence of finite dimensionally computable filters. In Stochastic Partial Differential Equations and Applications II: Proceedings of a Conference held in Trento, Italy February 1--6, 1988 , pp.\ 197--204. Springer

    2006

  104. [112]

    Capp \'e , R

    Olsson, J., O. Capp \'e , R. Douc, and \'E . Moulines (2008). Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models . Bernoulli\/ 14\/ (1), 155 -- 179

    2008

  105. [113]

    Willsky, and I

    Oppenheim, A., A. Willsky, and I. Young (1983). Signals and Systems . Prentice Hall

    1983

  106. [114]

    Nalisnick, D

    Papamakarios, G., E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan (2021). Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research\/ 22\/ (57), 1--64

    2021

  107. [115]

    Pardoux, E. (1981). Non-linear filtering, prediction and smoothing. In Stochastic Systems: The Mathematics of Filtering and Identification and Applications: Proceedings of the NATO Advanced Study Institute held at Les Arcs, Savoie, France, June 22--July 5, 1980 , pp.\ 529--557...

    1981

  108. [116]

    Pearlmutter, B. A. (1995). Gradient calculations for dynamic recurrent neural networks: A survey. IEEE Transactions on Neural Networks\/ 6\/ (5), 1212--1228

    1995

  109. [117]

    Peluchetti, S. and S. Favaro (2020). Infinitely deep neural networks as diffusion processes. In International Conference on Artificial Intelligence and Statistics , pp.\ 1126--1136. PMLR

    2020

  110. [118]

    Janati, and F

    Petetin, Y., Y. Janati, and F. Desbouvries (2021). Structured variational B ayesian inference for G aussian state-space models with regime switching. IEEE Signal Processing Letters\/ 28 , 1953--1957

    2021

  111. [119]

    Pontryagin, L. S., V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishechenko (1962). Mathematical theory of optimal processes. ZAMM - Journal of Applied Mathematics and Mechanics / Zeitschrift für Angewandte Mathematik und Mechanik\/ 43\/ (10-11), 514--515

    1962

  112. [120]

    Doucet, and S

    Poyiadjis, G., A. Doucet, and S. S. Singh (2011). Particle approximations of the score and observed information matrix in state space models with application to parameter estimation. Biometrika\/ 98\/ (1), 65--80

    2011

  113. [121]

    Narasimhan, T

    Radford, A., K. Narasimhan, T. Salimans, and I. Sutskever (2018). Improving language understanding by generative pre-training

    2018

  114. [122]

    Shazeer, A

    Raffel, C., N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research\/ 21\/ (140), 1--67

    2020

  115. [123]

    Rangapuram, S. S., M. W. Seeger, J. Gasthaus, L. Stella, Y. Wang, and T. Januschowski (2018). Deep state space models for time series forecasting. Advances in Neural Information Processing Systems\/ 31

    2018

  116. [124]

    Reinsel, G. C. (2003). Elements of Multivariate Time Series Analysis . Springer Science & Business Media

    2003

  117. [125]

    Shlezinger, X

    Revach, G., N. Shlezinger, X. Ni, A. L. Escoriza, R. J. Van Sloun, and Y. C. Eldar (2022). Kalman N et: Neural network aided kalman filtering for partially known dynamics. IEEE Transactions on Signal Processing\/ 70 , 1532--1547

    2022

  118. [126]

    Rezende, D. J., S. Mohamed, and D. Wierstra (2014). Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine Learning , pp.\ 1278--1286. PMLR

    2014

  119. [127]

    Roberts, G. O. and O. Stramer (2001). On inference for partially observed nonlinear diffusion models using the M etropolis- H astings algorithm. Biometrika\/ 88\/ (3), 603--621

    2001

  120. [128]

    Roesser, R. (1975). A discrete state-space model for linear image processing. IEEE transactions on automatic control\/ 20\/ (1), 1--10

    1975

  121. [129]

    Roth, M. and F. Gustafsson (2011). An efficient implementation of the second order extended K alman filter. In 14th International Conference on Information Fusion , pp.\ 1--6. IEEE

    2011

  122. [130]

    Rozovskii, B. L. (1991). A simple proof of uniqueness for K ushner and Z akai equations. In Stochastic Analysis , pp.\ 449--458. Elsevier

    1991

  123. [131]

    Rubanova, Y., R. T. Chen, and D. K. Duvenaud (2019). Latent ordinary differential equations for irregularly-sampled time series. Advances in Neural Information Processing Systems\/ 32

    2019

  124. [132]

    Shabani, and D

    Safari, S., F. Shabani, and D. Simon (2014). Multirate multisensor data fusion for linear systems using K alman filters and a neural network. Aerospace Science and Technology\/ 39 , 465--471

    2014

  125. [133]

    Sarkka, S. and J. Hartikainen (2010). On gaussian optimal smoothing of non-linear state space models. IEEE Transactions on Automatic Control\/ 55\/ (8), 1938--1941

    2010

  126. [134]

    Sch \"o n, T. B., F. Lindsten, J. Dahlin, J. W gberg, C. A. Naesseth, A. Svensson, and L. Dai (2015). Sequential M onte C arlo methods for system identification. IFAC-PapersOnLine\/ 48\/ (28), 775--786

    2015

  127. [135]

    Schorfheide, F. and D. Song (2015). Real-time forecasting with a mixed-frequency VAR . Journal of Business & Economic Statistics\/ 33\/ (3), 366--380

    2015

  128. [136]

    Shukla, S. N. and B. Marlin (2019). Interpolation-prediction networks for irregularly sampled time series. In International Conference on Learning Representations

    2019

  129. [137]

    Shukla, S. N. and B. Marlin (2020). Multi-time attention networks for irregularly sampled time series. In ICML Workshop on the Art of Learning with Missing Values (Artemiss)

    2020

  130. [138]

    Shumway, R. H. and D. S. Stoffer (1982). An approach to time series smoothing and forecasting using the EM algorithm. Journal of Time Series Analysis\/ 3\/ (4), 253--264

    1982

  131. [139]

    Shumway, R. H. and D. S. Stoffer (1991). Dynamic linear models with switching. Journal of the American Statistical Association\/ 86\/ (415), 763--769

    1991

  132. [140]

    Shumway, R. H. and D. S. Stoffer (2000). Time series analysis and its applications , Volume 3. Springer

    2000

  133. [141]

    De Mello, J

    Smith, J., S. De Mello, J. Kautz, S. Linderman, and W. Byeon (2024). Convolutional state space models for long-range spatiotemporal modeling. Advances in Neural Information Processing Systems\/ 36

    2024

  134. [142]

    Smith, J. T., A. Warrington, and S. Linderman (2023). Simplified state space layers for sequence modeling. In The Eleventh International Conference on Learning Representations

    2023

  135. [143]

    S rensen, H. (2004). Parametric inference for diffusion processes observed at discrete points in time: a survey. International Statistical Review\/ 72\/ (3), 337--354

    2004

  136. [144]

    Bobrovsky, and Z

    Steinberg, Y., B.-Z. Bobrovsky, and Z. Schuss (1994). Fixed-point smoothing of scalar diffusions I : An asymptotically optimal smoother. SIAM Journal on Applied Mathematics\/ 54\/ (3), 833--853

    1994

  137. [145]

    Stock, J. H. and M. W. Watson (2012). Dynamic factor models. The Oxford Handbook of Economic Forecasting\/

    2012

  138. [146]

    Stock, J. H. and M. W. Watson (2016). Dynamic factor models, factor-augmented vector autoregressions, and structural vector autoregressions in macroeconomics. In Handbook of Macroeconomics , Volume 2, pp.\ 415--525. Elsevier

    2016

  139. [147]

    Sun, Y., L. Dong, S. Huang, S. Ma, Y. Xia, J. Xue, J. Wang, and F. Wei (2023). Retentive network: A successor to transformer for large language models. arXiv preprint arXiv:2307.08621\/

    2023 arXiv

  140. [148]

    Tustin, A. (1947). A method of analysing the behaviour of linear systems in terms of time series. Journal of the Institution of Electrical Engineers-Part IIA: Automatic Regulators and Servo Mechanisms\/ 94\/ (1), 130--142

    1947

  141. [149]

    Tzen, B. and M. Raginsky (2019). Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit. arXiv preprint arXiv:1905.09883\/

    2019 arXiv

  142. [150]

    Shazeer, N

    Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, . Kaiser, and I. Polosukhin (2017). Attention is all you need. Advances in Neural Information Processing Systems\/ 30

    2017

  143. [151]

    Wan, E. A. and R. Van Der Merwe (2000). The unscented K alman filter for nonlinear estimation. In Proceedings of the IEEE 2000 adaptive systems for signal processing, communications, and control symposium (Cat. No. 00EX373) , pp.\ 153--158. Ieee

    2000

  144. [152]

    Shi, and S

    Wang, B., Z. Shi, and S. Osher (2019). Resnets ensemble via the F eynman- K ac formalism to improve natural and robust accuracies. Advances in Neural Information Processing Systems\/ 32

    2019

  145. [153]

    Tsepa, J

    Wang, C., O. Tsepa, J. Ma, and B. Wang (2024). Graph-mamba: Towards long-range graph sequence modeling with selective state spaces. arXiv preprint arXiv:2402.00789\/

    2024 arXiv

  146. [154]

    Smola, D

    Wang, Y., A. Smola, D. Maddix, J. Gasthaus, D. Foster, and T. Januschowski (2019, 09--15 Jun). Deep factors for forecasting. In K. Chaudhuri and R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning , Volume 97 of Proceedings of Machine ...

    2019

  147. [155]

    Woodbury, M. A. (1950). Inverting modified matrices . Department of Statistics, Princeton University

    1950

  148. [156]

    Wu, Y., D. Hu, M. Wu, and X. Hu (2006). A numerical-integration perspective on G aussian filters. IEEE Transactions on Signal Processing\/ 54\/ (8), 2910--2921

    2006

  149. [157]

    Heinonen, and H

    Yildiz, C., M. Heinonen, and H. Lahdesmaki (2019). ODE2VAE : Deep generative second order ODE s with B ayesian neural networks. Advances in Neural Information Processing Systems\/ 32

    2019

  150. [158]

    Yuan, Z., X. Ban, Z. Zhang, X. Li, and H.-N. Dai (2023). ODE - RSSM : learning stochastic recurrent state space model from irregularly sampled data. In Proceedings of the AAAI Conference on Artificial Intelligence , Volume 37, pp.\ 11060--11068

    2023

  151. [159]

    Zadeh, L. and C. Desoer (2008). Linear system theory: the state space approach . Courier Dover Publications

    2008

  152. [160]

    Zhu, L., B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang (2024). Vision M amba: Efficient visual representation learning with bidirectional state space model. In Forty-first International Conference on Machine Learning

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.