{"id":"40dcc932-0fb6-466e-ac31-f2f04eb488b4","arxiv_id":"2502.04518","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A Jordan-type LSTM that feeds the previous state estimate back into the network is shown to match Elman LSTM accuracy while training faster on three state estimation examples.","lead":"This paper introduces a Jordan-style LSTM network that feeds its previous state estimate back into the recurrent connections, and compares it with an Elman-style LSTM and Kalman filters for state estimation. The new network matches the Elman LSTM's estimation error on the test examples but trains in less time, and both LSTM variants beat the extended Kalman filter on nonlinear systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed JLSTM training-time advantage is confounded by unequal learning rates chosen to equalize final error; without matched-budget or repeated-run comparison, the central empirical claim is not established.","rationale":"The reader's weakest_assumption identifies the same confound: learning rates were chosen to make final errors comparable, and training times were then compared. I agree that this is the most load-bearing issue for the central claim. The KF baseline inconsistency is real but affects the weaker 'outperforms KF/EKF' component and is fixable by stating initial covariance; the universal approximation proof gap is a correctness issue, but the paper frames it as motivation and the empirical comparison is the actual claim. Because the claimed advantage is an empirical training-time benefit, a controlled, repeated-run comparison is the decisive check. The current CONDITIONAL verdict is appropriate: the concern does not prove the claim false, but it prevents acceptance as-is.","tokens_in":13578,"tokens_out":3831,"duration_ms":42620,"concrete_test":"On the reversed van der Pol example, retrain both JLSTM and ELSTM with a grid of learning rates (e.g., 1e-4, 3e-4, 1e-3, 3e-3, 1e-2), identical random seeds, same initialization scheme, same early-stopping patience, and a fixed validation split; report final NMSE, number of epochs, and wall-clock time for each configuration. If, at matched learning rates or on the best-tuned Pareto frontier of NMSE vs. training time, JLSTM is not consistently faster, the claim that the Jordan structure reduces training time is unsupported. Repeating with at least 5 seeds would also settle whether the single-run Table 2 entries are stable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that JLSTM is preferable to ELSTM because it reaches the same NMSE in considerably less training time. The only evidence is Table 2, but for each example the two architectures were trained with different learning rates, explicitly selected so that final NMSE values would be comparable (Section 5: pendulum 1e-4 vs 1e-3; van der Pol 1e-3 vs 1e-2; springs 1e-3 for both). A higher Adam learning rate can reduce the number of epochs needed to reach a given loss, so the van der Pol result (JLSTM 619 s vs ELSTM 3389 s) may be an artifact of the more favorable learning rate rather than the Jordan feedback. The two other examples show only 2-4% training-time differences, so the \"considerably smaller\" claim rests almost entirely on the one confounded comparison. No seeds, repeated runs, epoch counts, or error bars are reported, and training time is wall-clock, so it mixes epochs with per-epoch cost. If JLSTM's advantage disappears when learning rates are matched or when both architectures are tuned to their best achievable error, the central claim collapses. The universal approximation result has a recursion gap, but it is supporting motivation, not the load-bearing part of the empirical claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a Jordan-based long short-term memory (JLSTM) estimator for discrete-time state estimation, in which the previous state estimate, rather than the previous hidden state, is fed back into the gate and cell-input units. It states a universal approximation theorem for Jordan recurrent networks (Theorem 2.11), describes the ELSTM and JLSTM implementations, and compares them with the Kalman filter on a 20-state linear spring system and with the extended Kalman filter on a pendulum and a reversed van der Pol oscillator. The reported results claim that both LSTM filters achieve lower NMSE than the EKF on the nonlinear examples, that JLSTM matches ELSTM estimation error, and that JLSTM trains considerably faster than ELSTM.","tokens_in":13790,"tokens_out":6613,"duration_ms":72549,"significance":"If the claims were established, the paper would make a modest but useful contribution: a simple architectural variant that feeds the state estimate back into LSTM gates, with evidence that recurrent neural filters can outperform the EKF on low-order nonlinear systems and that the Jordan feedback structure can reduce training time. The paper's strengths are the use of standard benchmark examples, the inclusion of out-of-distribution initial-condition tests, and the transparent reporting of the network hyperparameters and training/test times. However, the central comparative claim is not yet supported: the training-time comparison is confounded by unequal learning rates chosen to equalize final errors, and the linear-system result appears inconsistent with Kalman optimality. The universal approximation theorem, although presented as supporting motivation, also has a proof gap. These issues are local and can be addressed with additional experiments and a revised proof, so the manuscript merits revision rather than rejection.","major_comments":[{"comment":"The proof of Theorem 2.11 establishes only a one-step uniform approximation of the map F on a compact set K, as in Eq. (7), and then asserts that continuity of F yields the recurrent bound ||\\bar{x}(t) - \\hat{x}(t)|| < epsilon. This does not follow: even if the network approximates F when evaluated at the same argument, the recurrent error can grow through the recursion unless a Lipschitz or contraction bound is imposed and an induction over t is carried out. In addition, K is assumed to contain both trajectories, but the network-generated trajectory \\hat{x}(t) cannot be assumed a priori to remain in K. As written, the theorem is unproved.","section":"Section 2, Theorem 2.11"},{"comment":"The paper's main conclusion, stated in Section 6 as 'Using a JLSTM instead of an ELSTM appears preferable because the JLSTM network has considerably smaller training time to achieve the same error,' rests on Table 2. For the down pendulum and reversed van der Pol examples, Sections 5.2 and 5.3 explicitly set different Adam learning rates for ELSTM and JLSTM (1e-4 versus 1e-3, and 1e-3 versus 1e-2) in order to make the final NMSE values comparable. A higher learning rate can reduce the number of epochs needed to reach a given loss, so the reported 5.5x training-time advantage on the van der Pol example cannot be attributed to the Jordan feedback structure. The other two examples show only 2-4% differences. No repeated runs, seeds, epoch counts, or error bars are reported, and training time is wall-clock time, which mixes epoch count with per-epoch cost. The authors should compare both architectures at matched learning rates, or report full training-time-versus-error curves over a range of learning rates with several random seeds.","section":"Section 5 and Table 2"},{"comment":"The connected-springs model is linear with Gaussian noises, for which the Kalman filter is optimal. Yet Table 1 reports NMSE 0.0153 for ELSTM and 0.0162 for JLSTM, both below the Kalman filter's 0.0174. Unless the KF was deliberately initialized with a mismatched covariance, or the finite test set makes the difference statistically insignificant, this contradicts KF optimality. Please clarify the KF initialization, report per-sequence variance or confidence intervals, and reconcile this result with the statement in Section 5.1 that 'theory shows the KF to be the best estimator for a linear system.'","section":"Table 1, connected springs"}],"minor_comments":[{"comment":"The first paragraph of Section 6 refers to 'a 100 dimensional linear system,' but the connected-springs example has a 20-dimensional state vector; please correct this.","section":"Section 6"},{"comment":"The quantity in Eq. (12) is called NMSE, but no normalization by the variance of the true signal is displayed; please define the normalization explicitly or rename the metric.","section":"Equation (12)"},{"comment":"The statement 'simple recurrent networks are a special case of long short term memory networks where the gates are not involved' is asserted without showing how the gate equations (8) and (9) reduce to the JRN/ERN recursions (2) and (3); a one-sentence construction would make the transfer of Theorem 2.11 to LSTMs transparent.","section":"Section 3"},{"comment":"The caption of Figure 2 mentions a bias 'by' that does not appear in Eq. (3); please clarify whether this is a typo or whether a bias term is missing from the forward-propagation equations.","section":"Figure 2"},{"comment":"The testing-time comparison in Table 3 may be unfair if the KF/EKF is implemented in Python on a CPU while the LSTM models run on a T4 GPU; please state the hardware and software used for all estimators.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the paper is on the borderline for a journal. The proposed JLSTM architecture is a straightforward variant of the LSTM, and the central empirical claim of faster training time is currently confounded by unequal learning rates. I would be willing to see a revision that includes matched-learning-rate training curves, repeated runs with error bars, a repaired proof of Theorem 2.11, and an explanation of the linear-system NMSE inconsistency. The topic is within the journal's scope, but the present evidence is insufficient for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a modest but real architecture variant — JLSTM feeds the previous state estimate into the LSTM gates instead of the previous hidden state — and on three low-order examples it matches Elman LSTM accuracy while sometimes training faster. That's new, though not deep.\n\nCredit where due: the update equations are clearly stated, the comparison against KF/EKF is standard, and the authors test on initial conditions outside the training region, which is a good check. The empirical direction (LSTM beats EKF on these nonlinear examples) is plausible and consistent with prior work.\n\nSoft spots, in proportion: the central claim — \"JLSTM trains considerably faster\" — rests entirely on Table 2. On the one example with a big gap (van der Pol, 619 vs 3389 seconds), the two networks were trained with different learning rates (1e-3 for ELSTM, 1e-2 for JLSTM). The authors chose rates to equalize final error, so the time comparison is confounded. On the other two examples, training times are within a few percent. Without matched learning-rate budgets or repeated runs, the speed advantage is not established.\n\nThe universal approximation proof (Theorem 2.11) jumps from a one-step function approximation to a trajectory bound without an induction or Lipschitz argument. That's a real gap, but it's supporting motivation, not the empirical claim.\n\nAlso, no code, data, seeds, or error bars are provided, so the numbers can't be checked. And the linear KF comparison is suspicious: LSTM NMSE below KF conflicts with KF optimality, suggesting a baseline mismatch or an unstated difference.\n\nNet: the paper is a reasonable workshop-level contribution or a starting point for a careful study. As is, the main claim is under-supported. If the authors fix the learning-rate confound, add repeated runs and release code, and tighten the proof, it becomes a modest but solid contribution.\n\nFor peer review: yes, send it to a referee. The idea is worth engaging with, and the flaws are fixable. I wouldn't cite it in its current form, but I'd watch for the revised version.","headline":"A modest but real LSTM variant whose claimed speed advantage is confounded by unequal learning rates; worth refereeing, not citing yet.","tokens_in":14341,"tokens_out":2221,"would_cite":false,"duration_ms":20430,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["93E11","68T07","93B07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A Jordan-style LSTM filters states as accurately as an Elman LSTM but trains far faster, and both beat the extended Kalman filter on nonlinear systems.","keywords":["state estimation","long short-term memory networks","Jordan recurrent neural networks","Elman recurrent neural networks","extended Kalman filter","nonlinear filtering","training time","universal approximation"],"falsifier":"Train both networks on the down-pendulum or van der Pol example using the same learning rate, the same early-stopping patience, and the same number of hidden units, then record epochs and NMSE; if JLSTM does not reach comparable error in fewer epochs, the paper's central claim about faster training is unsupported.","tokens_in":1694,"feed_emoji":"🧠","tokens_out":2648,"duration_ms":84565,"temperature":0.7,"pith_summary":"State estimation for nonlinear systems is hard because optimal filters require solving a difficult Hamilton-Jacobi-Bellman equation, so this paper asks whether recurrent neural networks can serve as practical estimators. It introduces a Jordan-style long short-term memory network (JLSTM) in which the previous state estimate, rather than the previous hidden state, is fed back into the LSTM gates, a structure that mirrors the dynamics of the system being estimated. The paper reports that on a ten-spring linear system and two nonlinear systems (down pendulum and reversed Van der Pol oscillator), the JLSTM achieves normalized mean-square error comparable to an Elman LSTM while taking substantially less training time, and both networks beat the extended Kalman filter on the nonlinear examples. The theoretical part shows that Jordan recurrent networks are universal approximators for a broad class of state estimators, and the paper positions the work as a step toward optimal estimation of high-order nonlinear systems.","feed_headline":"Jordan-style LSTM trains faster at equal error","feed_subtitle":"The Jordan-style feedback trains faster at equal error; both methods beat the extended Kalman filter on nonlinear systems.","key_machinery":"The central object is the JLSTM cell, defined by forward-propagation equations in which the four gate and cell-update operations use $\\hat{x}(t-1)$, the previous state estimate, in place of the previous hidden vector $a(t-1)$ used by an ELSTM. This recurrent connection type is inherited from Jordan recurrent neural networks, whose estimator form is $\\hat{x}(t+1)=W_{xa}\\sigma(W_{ay}y(t)+W_{ax}\\hat{x}(t)-b)$; the paper proves a universal approximation theorem showing that such recurrences can approximate any continuous state estimator of the form $\\bar{x}(t+1)=F(\\bar{x}(t),y(t))$. The architecture matters because it makes the network's recurrence conform to the structure of the discrete-time dynamical system, and that structural match is what the paper credits for faster loss minimization during training.","core_discovery":"The central discovery is empirical: changing the feedback path in an LSTM state estimator from hidden-state feedback (Elman) to previous-estimate feedback (Jordan) preserves estimation accuracy while reducing training time. For the three systems tested, the normalized mean-square errors of JLSTM and ELSTM are close, and the training time for JLSTM is smaller in every example (15983 versus 16298 seconds for the springs, 95644 versus 99473 for the pendulum, and 619 versus 3389 for the van der Pol oscillator). Both LSTM filters have lower NMSE than the extended Kalman filter for the nonlinear systems and NMSE comparable to the Kalman filter for the linear system. The authors conclude that using a JLSTM instead of an ELSTM appears preferable because the JLSTM reaches the same error with considerably less training, and they attribute the advantage to the Jordan structure's direct use of the previous state estimate, which resembles the discrete-time dynamics being estimated.","pith_inferences":["If the faster training is caused by the Jordan feedback rather than by unequal learning-rate settings, the same advantage should appear when both networks are trained with identical learning-rate schedules; this is a direct test of the architectural claim.","The structural match between the JLSTM and the state-space model suggests the architecture may extend to higher-order nonlinear systems, the paper's stated future goal, but this remains untested because the nonlinear examples are only second-order.","Because the JLSTM feeds back the state estimate itself, its recurrence resembles an observer, so existing observer theory could potentially yield convergence and stability guarantees rather than purely empirical accuracy.","The observation that both LSTM filters outperform the EKF outside the training range on nonlinear examples hints that learned estimators may be less sensitive to initial-condition mismatch than local linearization, which would matter for applications with uncertain starting states."],"forward_implications":["On the two nonlinear test systems, both JLSTM and ELSTM yield lower normalized mean-square error than the extended Kalman filter, suggesting that the nonlinear activation in LSTM cells reduces the linearization error that limits the EKF.","When initial conditions lie outside the training range, both LSTM estimators still beat the EKF on the nonlinear examples, so the learned filters appear to generalize beyond the training distribution in at least these cases.","For the linear spring system, JLSTM and ELSTM errors are close to the Kalman filter's error, while the KF remains the best estimator when testing outside the training range.","JLSTM requires substantially less training time than ELSTM across all three examples while reaching similar error, making it the preferable structure for these problems.","Testing time for both LSTM filters is much smaller than for the KF and EKF on the tested systems, with the LSTM filters taking 0.2 to 2.7 seconds versus 0.8 to 38.2 seconds for ten test sequences."],"supporting_citations":[{"why":"Provides the Kalman filter baseline for linear state estimation that the LSTM estimators are compared against.","marker":"Kalman (1960)"},{"why":"Introduces the Elman LSTM architecture for filtering and reports that it outperforms the EKF, the comparison this paper extends.","marker":"Cana, Herrero, and Lopez (2021)"},{"why":"Supplies the universal approximation result for Elman recurrent networks that the Jordan generalization builds on.","marker":"Schäfer and Zimmermann (2007)"},{"why":"Supplies the feedforward universal approximation theorem used in the proof that Jordan recurrences can approximate state estimators.","marker":"Hornik et al. (1989)"},{"why":"Prior work on stability of a simpler Jordan recurrent estimator that motivates the JLSTM structure.","marker":"Kaur et al. (2024)"},{"why":"Standard reference for the Kalman and extended Kalman filter implementations used as baselines.","marker":"Simon (2006)"},{"why":"Convergence analysis for learning in Jordan and Elman recurrent networks that supports training these structures.","marker":"Kuan, Hornik, and White (1994)"}],"fun_headline_variants":["JLSTM cuts training time, matches Elman accuracy","Jordan LSTM beats Elman on training speed for state estimation","Faster LSTM state estimation with Jordan feedback path","Same error, less training: Jordan LSTM state estimator","Jordan LSTM reduces training cost without error penalty"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The training-time comparison assumes that the different learning rates chosen for ELSTM and JLSTM, selected so the final errors would match, are fair choices, and that the faster convergence comes from the Jordan feedback rather than from the more favorable learning rate.","fun_headline_variants_meta":{"raw":{"variants":["JLSTM cuts training time, matches Elman accuracy","Jordan LSTM beats Elman on training speed for state estimation","Faster LSTM state estimation with Jordan feedback path","Same error, less training: Jordan LSTM state estimator","Jordan LSTM reduces training cost without error penalty"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000482,"raw_usage":{"total_tokens":2344,"prompt_tokens":870,"completion_tokens":1474,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":1396}},"tokens_in":486,"tokens_out":1474,"duration_ms":9228,"temperature":1.0,"reasoning_tokens":1396,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T22:26:23.117833+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train both networks on the down-pendulum or van der Pol example using the same learning rate, the same early-stopping patience, and the same number of hidden units, then record epochs and NMSE; if JLSTM does not reach comparable error in fewer epochs, the paper's central claim about faster training is unsupported.","supporting_citations":[{"cited_title":"APACrefauthors \\ 1960","cited_arxiv_id":null,"evidence_quote":"Provides the Kalman filter baseline for linear state estimation that the LSTM estimators are compared against."},{"cited_title":", Herrero, J G","cited_arxiv_id":null,"evidence_quote":"Introduces the Elman LSTM architecture for filtering and reports that it outperforms the EKF, the comparison this paper extends."},{"cited_title":", Stinchcombe, M","cited_arxiv_id":null,"evidence_quote":"Supplies the feedforward universal approximation theorem used in the proof that Jordan recurrences can approximate state estimators."},{"cited_title":", Zhou, R","cited_arxiv_id":null,"evidence_quote":"Prior work on stability of a simpler Jordan recurrent estimator that motivates the JLSTM structure."},{"cited_title":"APACrefauthors \\ 2006","cited_arxiv_id":null,"evidence_quote":"Standard reference for the Kalman and extended Kalman filter implementations used as baselines."},{"cited_title":", Hornik, K","cited_arxiv_id":null,"evidence_quote":"Convergence analysis for learning in Jordan and Elman recurrent networks that supports training these structures."}],"review_version":1}