{"id":"b8853fb1-52f9-448d-a0cb-c9da03542f21","arxiv_id":"2412.11211","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":1.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A selective review that unifies classical and deep learning state space models, from latent neural ODEs/SDEs to structured SSM architectures like S4 and Mamba.","lead":"This paper reviews deep learning approaches to state space models, covering VAE-based latent ODE/SDE models and efficient sequence architectures like S4 and Mamba. It is a useful map of a fast-moving field, though it introduces no new methods or results.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'unified perspective' rests on the claim that VAE is the de facto training pipeline, but the paper itself reviews multiple non-VAE neural SSMs and all Section 4 architectures; the taxonomy's organizing principle is internally inconsistent.","rationale":"The reader's weakest_assumption concerned whether the selected representative papers accurately represent the broader field. I agree that representativeness matters, but the more acute issue is internal: the paper's own reviewed works contradict the 'de facto standard' claim. The S4 generating-function derivation gap flagged by the reader is a real technical flaw, but it is localized to one formula in Section 4.2 and does not directly bear on the abstract's central claim of a unified perspective for Sections 2 and 3. The VAE-dominance claim, by contrast, is used as the organizing axis of the taxonomy: Section 2.3 introduces VAE as 'the de facto approach for training complex SSMs' and Section 6 reiterates that 'VAE [is] the dominant learning paradigm'. If a substantial share of the models the paper itself reviews are trained without VAE, the synthesis's central organizing principle is unreliable. This concern is concrete and testable, and it justifies the reader's CONDITIONAL verdict rather than ACCEPT. I do not see grounds to move to REJECT: the paper is a selective review, and a softened wording of the dominance claim would address the issue without invalidating the taxonomy. Hence the verdict remains CONDITIONAL, unchanged from the reader's assessment.","tokens_in":42103,"tokens_out":15519,"duration_ms":128380,"concrete_test":"Build a corpus of deep SSM papers from the reference lists of the review's own key citations (e.g., Chen et al. 2018, Krishnan et al. 2017, Gu et al. 2022, Hasan et al. 2021) together with papers citing them. For each, classify the training objective as VAE/ELBO, maximum likelihood/Kalman, adjoint/other. Count the fraction using VAE among models with neural-network parameterized state/observation equations. If the fraction is not clearly a majority, or if the non-VAE models are more recent and prominent, the 'de facto standard' claim and the unified perspective built on it need revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a unified perspective for discrete-time deep SSMs and continuous-time latent neural ODE/SDEs. The load-bearing assumption is that the VAE/ELBO pipeline is the de facto standard for training neural SSMs (Sections 2.3 and 6). The paper's own selection contradicts this: Section 3.3.1 states Jia and Benson (2019) 'does not rely on a VAE-based training pipeline' and instead uses the adjoint method; Deng et al. (2021) is presented without a VAE objective; Masti and Bemporad (2021) in Section 3.4 uses an autoencoder with reconstruction plus state-consistency losses, not an ELBO. Section 4 (S4, S5, Mamba) is a separate category of deterministic SSM modules trained with standard sequence objectives, not VAE. If the taxonomy's main axis is 'partially DL with classical filtering' versus 'fully DL with VAE', then these non-VAE fully DL models have no place, and the claimed unified perspective overstates VAE's role. This is not merely an external completeness issue; it is an internal inconsistency between the assertions in Sections 2.3 and 6 and the reviewed models. The synthesis would mislead readers about what unifies the field.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper is a selective review of deep learning-based approaches to state space models (SSMs). It aims to present a unified perspective that covers discrete-time deep SSMs, continuous-time latent neural ODE/SDEs, and recent SSM-derived sequence-modeling architectures (LSSL, S4, S5, Mamba). The review begins with classical maximum-likelihood-based learning, including filtering/smoothing and the EM algorithm, then describes the variational autoencoder (VAE) pipeline as the dominant learning framework for latent-variable neural SSMs. It subsequently surveys representative deep SSM papers organized by discrete-time, latent ODE, and latent SDE formulations, discusses additional related approaches, and closes with applications to mixed-frequency and irregularly-spaced time series. Appendices provide background on Kalman filtering, nonlinear filters, and SDE-based SSMs.","tokens_in":42363,"tokens_out":3817,"duration_ms":35663,"significance":"If the synthesis were accurate, this review would be a useful bridge between two largely separate literatures: the statistical latent-variable SSM tradition and the modern deep learning sequence-modeling tradition. The paper covers a broad range of material in a clear, well-organized manner, and its derivations of the ELBO, filtering recursions, and the LSSL/S4 machinery are mostly standard and readable. The review also has the merit of making the connections between classical filtering and modern encoder-decoder pipelines explicit. However, the central claim of a unified perspective is weakened by an internal inconsistency in the taxonomy: the paper asserts that VAE-based training is the de facto standard for deep SSMs, yet it reviews several non-VAE neural SSMs without reconciling them with that claim. A corrected and more carefully qualified taxonomy would make this a valuable review for researchers entering the field.","major_comments":[{"comment":"The assertion that VAE-based training is 'the de facto approach' (Section 2.3) and 'the dominant learning paradigm' (Section 6) is contradicted by models reviewed elsewhere in the same paper. Section 3.3.1 explicitly states that Jia and Benson (2019) 'does not rely on a VAE-based training pipeline' and instead uses the adjoint method; Deng et al. (2021) is presented without a VAE objective; Masti and Bemporad (2021) in Section 3.4 uses an autoencoder with reconstruction plus state-consistency losses, not an ELBO; and Section 4 models (S4, S5, Mamba) are deterministic SSM modules trained with standard sequence objectives. Since the taxonomy in Section 2.3 is organized around the 'partially DL vs fully DL with VAE' distinction, these non-VAE models fall outside its organizing principle, making the claimed unified perspective overstate VAE's role. Please revise the claim to a more qualified statement, e.g., 'VAE is the dominant paradigm in the latent generative strand,' or provide a tripartite taxonomy that accommodates the non-VAE fully DL models.","section":"Section 2.3 and Section 6"},{"comment":"The derivation of the generating function identity is mathematically incorrect. The finite geometric sum formula gives sum_{ell=0}^{L-1} C \\bar{A}^ell \\bar{B} \\phi^ell = C(I - \\bar{A}^L \\phi^L)(I - \\bar{A}\\phi)^{-1}\\bar{B}. The last equality in footnote 11, which drops the factor (I - \\bar{A}^L \\phi^L), holds only if \\bar{A}^L = I, not merely because \\phi^L = 1. Since \\bar{A} is a general structured matrix, the asserted identity in (4.5) does not follow. This is not a minor typographical issue: it misrepresents the S4 technique, which is one of the central topics of Section 4. Please correct the presentation, for instance by stating the finite-sum identity with the correction term and explaining how the S4 algorithm handles it, or by deferring to the original derivation in Gu et al. (2022).","section":"Section 4.2, Eq. (4.5) and footnote 11"},{"comment":"The classification of Masti and Bemporad (2021) as falling under the 'fully DL' category is inconsistent with the definition in Section 2.3, which states that fully DL approaches learn parameters with a VAE-based pipeline. The loss described in Section 3.4 is a sum of a reconstruction error and a state-consistency discrepancy, with no ELBO or variational posterior. This is an internal inconsistency, not merely an omission: the paper's own taxonomy assigns the model to a category that its definition does not fit. Please either reclassify the method or amend the definition of the fully DL category to include non-VAE training objectives such as reconstruction-plus-consistency losses.","section":"Section 3.4 and Section 2.3"}],"minor_comments":[{"comment":"The sentence 'Detailed explanations if these two approaches can be found in the respective references' contains a typo: 'if' should be 'of'.","section":"Section 5.2"},{"comment":"The notation '(\\Delta t) Id' is ambiguous; it should be '(\\Delta t) I_d' to indicate the d-dimensional identity matrix.","section":"Section 3.3, Eq. (3.10)"},{"comment":"The KL divergence terms contain a duplicated vertical bar, 'KL(q(.) || p(.))' is written with two pairs of parallel bars; one pair should be removed for clarity.","section":"Section 2.3, Eq. (2.10)"},{"comment":"The phrase '~C collects the constant term' is vague. It would be clearer to state that the derivation incorrectly treats the truncated sum as if it were an infinite sum; the intended constant term appears after using \\phi^L = 1 in a corrected finite-sum identity.","section":"Section 4.2, footnote 11"},{"comment":"The description of the forecasting variant for Rubanova et al. (2019) partitions the observed time points into two halves, but it is not specified whether the split is by time index or by sorted timestamps; please clarify.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"This is a review paper rather than an original research contribution, so the correctness of its expository equations and the internal consistency of its taxonomy are particularly important. The two main issues—the overstated VAE-centric taxonomy and the incorrect S4 generating-function identity—are both fixable within the manuscript's scope. I recommend major revision: the authors should correct the mathematical derivation in Section 4.2 and substantially qualify or restructure the VAE-‘de facto standard’ claims. If these are addressed, the paper would likely be suitable for publication as a selective review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth reading if you want a compact map of deep state-space models, but the stress-test concern is on target: the central framing overstates the role of VAE, and the S4 derivation contains a genuine mathematical slip.\n\nWhat the paper does well: it organizes a fast-moving literature into useful buckets—discrete-time deep SSMs, latent neural ODEs/SDEs, and SSM-as-module architectures (S4/S5/Mamba). The classical background on Kalman filtering, smoothing, and SMC is careful and well presented, and the appendices on Gaussian filters and nonlinear SDE filtering are thorough. For a graduate student entering the field, this is a legitimate starting point.\n\nThe soft spots are real but addressable. First, the abstract and Section 6 claim a unified perspective built on the VAE pipeline being the de facto standard for training neural SSMs. Yet the paper's own Section 3.3.1 reviews Jia and Benson (2019) and Deng et al. (2021), which are fully deep but not VAE-trained; Masti and Bemporad (2021) uses an autoencoder loss, not an ELBO; and Section 4 is explicitly a separate enterprise trained with standard sequence losses. So the taxonomy's organizing axis is internally inconsistent—the paper says fully-DL equals VAE, then includes fully-DL non-VAE models without reclassification. This is not a desk-rejectable flaw, but the framing needs an honest revision, say 'a common pipeline' rather than 'the unifying framework'.\n\nSecond, Equation (4.5) and footnote 11 misstate the finite geometric sum identity. The equality K_L(phi) = C(I - A^L phi^L)(I - A phi)^{-1}B does not reduce to C(I - A phi)^{-1}B merely because phi^L = 1; the factor (I - A^L) remains. You need an additional argument (spectral radius or L to infinity), and as written the derivation is wrong. A reader using this review as a technical reference would be misled.\n\nBottom line: this is a solid organizational contribution with two correctable defects. It deserves peer review—a serious referee can push for the framing fix and the equation fix. I would not cite it as a technical reference in its current form, but I would point students to it as a survey after revision.","headline":"Useful selective review of deep SSMs, but its VAE-centric framing contradicts its own surveyed works, and the S4 derivation has a real gap.","tokens_in":42857,"tokens_out":3125,"would_cite":false,"duration_ms":28573,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62M10","62M20","68T07","60H10"],"pacs":[],"model":"deepseek-v4-flash","headline":"A selective review argues that deep state-space models—from neural Kalman filters through latent ODEs/SDEs to S4 and Mamba—fit one unified template.","keywords":["state space models","variational autoencoders","neural ODEs","neural SDEs","structured state space models","sequence modeling","mixed frequency time series","irregularly-spaced time series"],"falsifier":"One concrete test: assemble the population of neural SSM papers since 2021 and record the training objective. If a sizable share of prominent models are trained without any ELBO—by direct likelihood, Kalman-style marginalization, or simulation-based inference—the 'de facto standard' claim fails; if a major line fits neither the generative latent-model branch nor the standalone-layer branch, the unified taxonomy fails.","tokens_in":1809,"feed_emoji":"🧠","tokens_out":2559,"duration_ms":88102,"temperature":0.7,"pith_summary":"The paper sets out to show that recent deep-learning state-space models are variations on one template rather than separate inventions. The template is a two-equation structure: a latent state process that evolves in discrete or continuous time, and an observation map that links states to data, learned through a variational autoencoder that encodes an approximate posterior and decodes reconstructions. The same equations reappear as standalone neural-network layers for efficient long-sequence modeling, where the technical work is about making the state-transition matrix structured so recurrence, convolution, and parallel scan all become cheap. A sympathetic reader would care because the synthesis gives a common vocabulary for comparing neural Kalman filters, latent neural ODEs/SDEs, and modern sequence layers such as S4, S5, and Mamba.","feed_headline":"One template unifies neural state-space models","feed_subtitle":"A review maps deep Kalman filters, latent ODEs/SDEs, and long-sequence layers like S4 and Mamba onto one framework.","key_machinery":"The load-bearing object is the two-equation state-space template, $z_t = f(z_{t-1}, u_t)$ and $x_t = g(z_t, \\epsilon_t)$ (or their continuous-time counterparts), together with the learning pipeline built on the variational evidence lower bound (ELBO). The ELBO splits into reconstruction terms and KL-divergence terms between the encoder and the prior or state-transition distributions; this decomposition is what lets an encoder network stand in for filtering and smoothing. In the architectural branch, the key objects are the linear state-space layer with state matrix $A$, input map $B$, output map $C$, and timescale $\\Delta$, the structured HiPPO initialization that gives the state long memory, and the diagonal-plus-low-rank parameterization, generating-function or parallel-scan algorithms, and input-dependent timescale selection that turn exponential cost into near-linear cost.","core_discovery":"The paper's central claim is that a unified perspective exists for discrete-time deep state-space models and continuous-time latent neural ODE/SDE models. All of them share the same skeleton of a state equation plus an observation equation, and they differ mainly in whether time is discrete or continuous and in how the posterior over latent states is approximated. The review further asserts that VAE-based training—an encoder that approximates the posterior and a decoder that reconstructs observations from sampled states—has become the de facto standard for deep generative SSMs, replacing explicit filtering and smoothing recursions where those are intractable. On the architectural side, it treats linear state-space layers and their structured successors as the same mathematical object used as a building block for efficient sequence modeling, with HiPPO-based initialization, diagonal-plus-low-rank parameterization, and selective input-dependent timescales as successive refinements.","pith_inferences":["Beyond the paper, the same state-plus-observation template could serve as a benchmark testbed: instantiate the same encoder and observation map while swapping discrete, ODE, SDE, and selective state dynamics, isolating what each state-equation choice contributes to accuracy and efficiency.","The paper's two-branch split into generative latent model versus standalone layer is probably already blurring, since architectural SSMs are being connected back to attention and to input-dependent latent dynamics; a future unified account may treat Transformers and SSMs as endpoints of the same continuum.","A testable extension of the 'VAE as de facto standard' claim is to count training objectives in recent neural-SSM publications; if direct likelihood estimation or simulation-based inference occupies a nontrivial share, the standard would need qualification."],"forward_implications":["Classical filtering and smoothing and deep VAE training are two routes to the same posterior-over-latent-states problem, so ideas like smoothing can inform encoder design in deep SSMs, and vice versa.","Latent neural ODEs and SDEs occupy a continuum with discrete-time deep SSMs: discrete models become ODE-like as step sizes shrink, and SDEs add a diffusion term, with the different encoder and decoder layouts forced by how the posterior factorizes over time.","In the architectural line, the same linear state-space equations support three equivalent views—recurrence, convolution, and continuous-time dynamics—so S4, S5, and Mamba can be trained in parallel and generated from sequentially by switching representations.","For mixed-frequency and irregularly-spaced data, the SSM formulation has a structural advantage because the latent state lives on a common or continuous time axis, letting one model produce forecasts, nowcasts, imputations, and extrapolations without retraining per vintage.","If the unified perspective holds, researchers can transfer algorithmic advances across branches, for example using structured-matrix tricks from the architectural line to speed up inference in generative latent SSMs."],"supporting_citations":[{"why":"Supplies the VAE/ELBO pipeline that the review identifies as the standard learning mechanism for deep generative SSMs.","marker":"Kingma and Welling (2013)"},{"why":"Establishes the discrete-time deep SSM template with Gaussian state transitions and structured inference networks.","marker":"Krishnan et al. (2015, 2017)"},{"why":"Introduces neural ODEs and the adjoint method that latent ODE and SDE models build on.","marker":"Chen et al. (2018)"},{"why":"Introduces latent neural ODEs and their VAE-based training pipeline for irregularly-sampled time series.","marker":"Rubanova et al. (2019)"},{"why":"Anchors the latent neural SDE review, including the encoder approximation and the isotropic-diffusion simplification.","marker":"Hasan et al. (2021)"},{"why":"Introduces S4 and the diagonal-plus-low-rank parameterization that makes linear SSM layers computationally efficient.","marker":"Gu et al. (2022)"},{"why":"Introduces S5, the MIMO formulation, and parallel scan, connecting SSM layers to recurrence in the time domain.","marker":"Smith et al. (2023)"},{"why":"Introduces Mamba's selective input-dependent parameterization as the current endpoint of the architectural SSM line.","marker":"Gu and Dao (2023)"}],"fun_headline_variants":["Unified framework ties discrete and continuous deep SSMs","VAE training emerges as standard for neural state-space models","Deep SSMs: from latent ODEs to sequence-model building blocks","One skeleton for Kalman, ODE, and Mamba-style models"],"cache_read_input_tokens":45056,"weakest_assumption_plain":"The synthesis rests on the selection of representative papers: if the chosen examples do not faithfully represent the broader literature, the claimed unity and the 'de facto' status of VAE training could be selection artifacts.","fun_headline_variants_meta":{"raw":{"variants":["Unified framework ties discrete and continuous deep SSMs","VAE training emerges as standard for neural state-space models","Deep SSMs: from latent ODEs to sequence-model building blocks","One skeleton for Kalman, ODE, and Mamba-style models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000257,"raw_usage":{"total_tokens":1549,"prompt_tokens":885,"completion_tokens":664,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":592}},"tokens_in":501,"tokens_out":664,"duration_ms":5614,"temperature":1.0,"reasoning_tokens":592,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:10:14.909082+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete test: assemble the population of neural SSM papers since 2021 and record the training objective. If a sizable share of prominent models are trained without any ELBO—by direct likelihood, Kalman-style marginalization, or simulation-based inference—the 'de facto standard' claim fails; if a major line fits neither the generative latent-model branch nor the standalone-layer branch, the unified taxonomy fails.","supporting_citations":[{"cited_title":"Shalit, and D","cited_arxiv_id":null,"evidence_quote":"Establishes the discrete-time deep SSM template with Gaussian state transitions and structured inference networks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces latent neural ODEs and their VAE-based training pipeline for irregularly-sampled time series."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces S5, the MIMO formulation, and parallel scan, connecting SSM layers to recurrence in the time domain."}],"review_version":1}