Pith. sign in

REVIEW 4 major objections 5 minor 94 references

A two-parameter nearest-neighbor map matches or beats foundation models at zero-shot reconstruction of chaotic and cyclic dynamical systems.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:39 UTC pith:YW4XJZEE

load-bearing objection DynaBase is a clever theoretical distillation of DSR foundation models, but its zero-shot performance claim currently rests on unverified inherited baseline numbers and a context-length inconsistency. the 4 major comments →

arxiv 2607.14937 v1 pith:YW4XJZEE submitted 2026-07-16 cs.LG cs.AImath.DSnlin.CD

A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems

Christoph J\"urgen Hemmer , Florian Plaswig , Daniel Durstewitz This is my paper
classification cs.LG cs.AImath.DSnlin.CD MSC 37M1037D4568T07
keywords dynamical systems reconstructionzero-shot forecastingin-context learningnearest-neighbor methodschaosinterpretable machine learningfoundation modelspiecewise-linear maps
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that zero-shot reconstruction of chaotic and cyclic dynamical systems does not require a large pretrained foundation model. By stripping a state-of-the-art mixture-of-experts model down to its bare operation, the authors arrive at DynaBase: a two-parameter map that, at each step, looks up the closest point in the provided context time series and linearly blends the current state with that point and its temporal successor. They show this minimal map matches or beats far larger models on long-term attractor statistics, and that its single blending coefficient α interpolates between mere context replay (α=0), flow mimicry (α=1), and bounded chaos (α>1). The payoff, if true, is a fully interpretable account of what in-context learning for dynamics actually needs.

Core claim

DynaBase defines forecasts by z_{t+1}=α z_t + β c_{τ(z_t)} + (1−α−β) c_{τ(z_t)+1}, where τ(z) selects the context point nearest to the current state. A symmetry argument (Theorem 1) shows any nearest-neighbor affine map that respects rotations and translations must have this form. The paper proves that at α=0 the map exactly reproduces the context-parroting algorithm, that α in [0,1] keeps orbits bounded near the context, and that for α>1 the map can produce genuinely chaotic, bounded attractors. Empirically, the single coefficient α trained on a small set of chaotic systems converges near 1.01, and this fixed value reconstructs 54 unseen systems with accuracy comparable to custom-trained an

What carries the argument

The recursive affine nearest-neighbor map: at each step the state looks up its nearest neighbor c_s in the context and maps to α z + β c_s + γ c_{s+1} with α+β+γ=1. The nearest-neighbor selector induces a Voronoi tessellation inside which the map is affine with Jacobian αI, so one scalar controls local expansion and contraction; the context supplies the vector field.

Load-bearing premise

The context must densely sample the attractor region where the test trajectory travels; if the context is sparse, noisy, or does not cover that region, the nearest neighbor's successor is not a good model of the local dynamics, and the paper's own Figure 4b shows failure outside the context-covered region.

What would settle it

Take a chaotic system whose attractor is sampled sparsely (short context, e.g., 200 steps) or add noise to the context, then run DynaBase with the proposed α≈1.01 and measure Dstsp against the true attractor; if long-term geometry degrades sharply relative to the dense-context case, the claim that one scalar plus context suffices is bounded.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The minimal requirement for zero-shot DSR is a context lookup plus one scalar; large pretrained architectures may be overkill.
  • Training objective matters: one-step MSE collapses to context parroting (α<1) and misses chaos, while DSR-specific training recovers α>1 and correct Lyapunov exponents.
  • Context parroting, observed in transformer time-series models, is a special case (α=0) that cannot represent true limit cycles or chaos.
  • A universal α≈1.01 could serve as a parameter-free forecaster: fix α once and only supply the context.
  • Closed-form least-squares fitting means model fitting is essentially free compared with foundation-model pretraining.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the reduction generalizes, it suggests that other 'emergent' in-context capabilities in time-series transformers may reduce to nearest-neighbor-style retrieval plus a linear readout.
  • The Voronoi-cell picture implies a testable scaling: reconstruction quality should degrade as context density decreases or as the attractor's dimension approaches the observation dimension; delay embedding may recover some of this.
  • One could investigate whether α≈1.01 is a universal constant across broader DS families or an artifact of the Lorenz-like systems in the benchmark; measuring the optimal α on systems with strongly non-uniform attractor geometry would settle this.
  • The map's continuous-time limit (Equation 15) might allow connecting the discrete nearest-neighbor rule to conventional ODE models and Lyapunov spectrum analysis via saltation operators.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DynaBase, a minimal two-parameter recursive map z_{t+1} = α z_t + β c_{τ(z_t)} + (1−α−β) c_{τ(z_t)+1}, obtained by iteratively simplifying the DynaMix foundation model. It claims that this nearest-neighbor affine recurrence achieves highly competitive zero-shot dynamical-system reconstruction across 54 test systems, that its one-parameter self-consistent version recovers context parroting at α=0 and can produce bounded chaos for α>1, and that closed-form least-squares fitting and grid search on a DSR objective yield interpretable parameter regimes. The theoretical results in the appendix are clearly stated and largely self-contained. The empirical comparison, however, relies on baseline numbers taken from the authors' own prior DynaMix paper and contains an unresolved context-length inconsistency between Fig. 1, Appx. A.1, and Appx. A.4.

Significance. If the empirical claims survive a matched-protocol comparison, the paper would make a substantial contribution: it would show that the minimal mechanism for zero-shot DSR is a nearest-neighbor lookup plus one effective scalar, and it would explain previously divergent observations about context parroting and chaotic reconstruction. The theoretical portion is a genuine strength: Theorems 1–4 are clean, the derivation of context parroting from the α=0 limit is exact, and the bounded-chaos example in Theorem 4 is explicit. The closed-form linear-regression estimator and the parameter-free zero-shot variant are also attractive. However, the headline 'highly competitive' claim is currently not independently verifiable because all non-DynaBase numbers in Fig. 1 are imported from [43] and the manuscript states conflicting context lengths. The paper's own Limitations paragraph appropriately narrows the scope, but the empirical load-bearing part must be fixed before the central claim can be accepted.

major comments (4)
  1. [Fig. 1 caption vs. Appx. A.1/A.4] The central claim that DynaBase is highly competitive rests on Fig. 1, but the manuscript gives inconsistent context lengths. Fig. 1 caption says 'same context length TC=2000'; Appx. A.1 says 'For the experiments in Fig. 1, we set T=2000 and TC=1000'; Appx. A.4's Fig. 5 caption says 'context window of TC=2000'. Since eq. (10) predicts the successor of the nearest context point, performance depends directly on context density and coverage. If DynaBase was evaluated with TC=1000 while imported baselines used TC=2000 (or vice versa), the comparison is unmatched. State the exact context length used for each model and rerun the comparison under a single protocol.
  2. [Appx. A.1 (Comparison methods)] All non-DynaBase baselines in Fig. 1 are listed as 'Evaluations were taken from [43]' and are not recomputed. The manuscript does not demonstrate that [43] used the same context construction, training/context split, or initialization as used here for DynaBase. Because [43] is also the source of the starting model, the competitive claim is not independently established. Rerun at least the key baselines (context parroting, DynaMix, Chronos) under the exact DynaBase evaluation protocol, or provide a detailed equivalence argument and reproduce the reported numbers.
  3. [Appx. A.1 (Grid search) / Sec. 4.2] The grid-search training is restricted a priori to the self-consistency line beta=-alpha: Appx. A.1 states 'sweep across alpha in [0,1.2] with step size delta_alpha=0.002 along the self-consistency line beta=-alpha'. Section 4.2 then invokes Fig. 2b as evidence that 'optimal DSR solutions essentially lie along a 1-parameter curve'. This is partially circular: the constraint is imposed before fitting. Show the full two-parameter loss landscape and quantify the width of the valley; otherwise the one-parameter reduction is an assumption rather than a finding.
  4. [Sec. 3.3, Eq. (11); Limitations] The reduction to the one-parameter map f_alpha assumes the exact self-consistency condition c_{i+1}=f(c_i) for all context points. This holds only for noiseless observations. The only noise experiment (Fig. 14) is qualitative, and the Limitations paragraph states that noise was not explicitly considered. Since observational noise breaks eq. (11) and destabilizes the nearest-neighbor lookup tau(.), please state how the results—especially the alpha=1.006 zero-shot estimate—degrade under noise, or explicitly restrict the central claims to noiseless contexts.
minor comments (5)
  1. [Sec. 3.4] In eq. (13), the notation mixes z_t and c_{τ(z_t)+1}; the design matrix in Appx. A.1 uses z_i and c_{τ(z_i)+1}. Please unify indexing.
  2. [Fig. 2b] The loss landscape panel lacks a colorbar or numeric scale; add one to allow quantitative interpretation of the valley.
  3. [Sec. 4.2] The term 'parameter-free zero-shot forecaster' is misleading: alpha=1.006 is estimated on a separate training set, so it is fixed rather than parameter-free. Suggest 'no per-DS fitting'.
  4. [General] The text repeatedly renders 'Voronoi' with a stray space ('V oronoi'), which should be fixed in the final version.
  5. [Appx. A.2] In the MASE definition, the symbol n appears both as the forecast horizon in the numerator and in the denominator scaling; please distinguish n from the total training length.

Circularity Check

0 steps flagged

No circular reduction: DynaBase's theoretical chain is self-contained; the inherited [43] baselines and a context-length ambiguity are verification concerns, not circularity.

full rationale

The derivation of DynaBase is self-contained. Sec. 3.2-3.3 reduce DynaMix by explicit ablations and prove Theorem 1 in Appx. B; the two-parameter form Eq. 10 and the one-parameter self-consistent form Eq. 12 follow from the stated equivariance assumptions and the self-consistency condition Eq. 11 (which algebraically forces gamma=1 for non-constant context). Theorems 2-4 are proved from the map's definition; Theorem 2 simply identifies the alpha=0 special case with the [92] context-parroting algorithm, a deliberate special case rather than a fitted prediction. The zero-shot alpha=1.006 is estimated on a separate training set, not on the test contexts. The only self-reference is that Fig. 1 baseline values are 'taken from [43]' (the authors' prior work), and Appx. A.1 says TC=1000 while Fig. 1 says TC=2000. These are reproducibility and matched-comparison concerns, not a reduction of any predicted quantity to its inputs; no fitted parameter is renamed as a prediction.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The central model has exactly two continuous free parameters (α,β) plus the chosen context length. No new physical entities are introduced; Voronoi cells and saltation operators are standard tools from the cited literature. The main load-bearing assumptions are the symmetry assumptions and the noiseless self-consistency condition that reduce the model to one parameter.

free parameters (2)
  • α = zero-shot: 1.006 (estimated on separate training set); per-system LS/DSR fits vary; grid searches over [0,1.2]
    Mix weight of the current state; controls local con/divergence and determines whether the map is parroting, cyclic, or chaotic. Estimated by OLS or grid search on Dstsp; the 'parameter-free zero-shot' still uses a fitted α, so it is not strictly parameter-free.
  • β = per-system values from OLS; set to −α in the self-consistent 1-parameter reduction
    Mix weight of the nearest context neighbor, with γ constrained by α+β+γ=1. Under the self-consistency condition eq. 11, β=−α, reducing the model to one parameter.
axioms (3)
  • domain assumption O(N)-equivariance and translation equivariance of the reconstruction (Thm 1, Appx. B.1)
    The uniqueness theorem assumes the flow map commutes with rotations/reflections and translations in state space. This is natural in DSR because coordinate changes do not alter dynamics, but it is an assumption rather than a derived fact.
  • domain assumption Self-consistency: c_{i+1}=f(c_i) for all context points (eq. 11)
    Used to reduce the two-parameter form to the 1-parameter family and to justify the grid-search line β=−α. Holds exactly only for noise-free, exactly sampled dynamics; the paper provides empirical loss-landscape support but does not prove it in noisy settings.
  • domain assumption Context densely samples the attractor so the nearest-neighbor successor approximates the local vector field (Sec. 3.2, eqs. 7-10; Sec. 4.3)
    The entire predictive mechanism relies on the successor of the nearest context point being a good local estimate. Fig. 4b demonstrates failure when the test state lies outside the context-covered region.

pith-pipeline@v1.3.0-alltime-deepseek · 23001 in / 18914 out tokens · 176834 ms · 2026-08-02T00:39:31.478975+00:00 · methodology

0 comments
read the original abstract

Recent foundation models (FMs) for zero-shot reconstruction of dynamical systems (DS) achieve strong out-of-domain generalization but provide little insight into the mechanisms that underlie their forecasts. Such an understanding could help to strip down overladen FM architectures to their bare essence and expose the minimal requirements for in-context learning in the DS domain. Toward this goal, here we iteratively reduce a recent powerful SOTA model for DS reconstruction, DynaMix (Hemmer & Durstewitz, 2025), to a minimal interpretable two-parameter form, which we call DynaBase. DynaBase produces forecasts through a linear blend of the current latent state and the nearest in-context neighbor and its temporal successor. Surprisingly, despite its extreme simplicity, DynaBase produces highly competitive zero-shot DS reconstructions across chaotic and cyclic systems, with a negligible parameter load, many orders of magnitude below that of other FMs. Even more, this extreme simplicity permits direct model optimization on DS reconstruction measures, as well as closed-form one-step analytical solutions on prediction MSE. Theoretical and empirical analysis of DynaBase further leads to a 1-parameter family of maps, with the context-parroting algorithm of (Zhang & Gilpin, 2026) recovered at one end, and chaotic (divergent but bounded) behavior at the other. We further show how different training strategies lead to models either optimal for short-term prediction or for DS reconstruction. Thus, DynaBase not only exposes the minimal mechanisms required for producing zero-shot DS reconstruction, but also reconciles within an accessible mathematical frame divergent observations in the literature.

Figures

Figures reproduced from arXiv: 2607.14937 by Christoph J\"urgen Hemmer, Daniel Durstewitz, Florian Plaswig.

Figure 1
Figure 1. Figure 1: DSR performance across all 54 test DS used in [43]. Minimal models (red: DynaBase training variants and pure context parroting), custom-trained DSR models (gray), and pretrained zero-shot foundation models including a zero-shot variant of DynaBase (blue) are compared for the same context length TC = 2000 (values for custom-trained and foundation models taken from [43]). Median±MAD of Dstsp (left, geometric… view at source ↗
Figure 2
Figure 2. Figure 2: a) Comparison of distributions of estimated αˆ values across all 54 chaotic test DS [43] for the two training objectives (sect. 3.4). Closed-form minimization of the 1-step MSE yields α < 1, outside of the chaotic regime. In contrast, optimization for DSR places most α values firmly (and correctly) into the chaotic regime, with a median αˆ ≈ 1.01. b) DSR loss landscape (optimization for Dstsp) across (α, β… view at source ↗
Figure 3
Figure 3. Figure 3: DynaBase’ loss landscape reflects the dynamics of the target system when trained for DSR. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: a) Voronoi tessellation for the periodic Selkov DS induced by the nearest-neighbor lookup τ (·) in DynaBase. The context C1:TC partitions the state space into cells Vs = {z : τ (z) = s}, within which DynaBase is described by a strictly affine map zt+1 = α zt + c˜s. b) Out-of-context forecasting on the 2d Selkov system. When the initial condition lies outside of the context-covered region, neither parroting… view at source ↗
Figure 5
Figure 5. Figure 5: DSR test set of 54 different DS gathered from [ [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Zero-shot reconstruction of chaotic Lorenz-63 system using DynaBase with [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Zero-shot reconstruction of chaotic Sprott-F system using DynaBase with [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Zero-shot reconstruction of chaotic Sprott-D system using DynaBase with [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Zero-shot reconstruction using α = 1 of cyclic Lorenz-63 system using DynaBase. Left: State space, center: time graphs, right: power spectrum. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Illustration of context parroting and flow mimicry. [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Lyapunov exponents estimated via the Rosenstein algorithm (see Appx. [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Different reconstructions by DynaBase along a bifurcation diagram of the Lorenz-63 [PITH_FULL_IMAGE:figures/full_fig_p021_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: DynaBase forecasting of a chaotic laser system [ [PITH_FULL_IMAGE:figures/full_fig_p022_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: DynaBase reconstruction of Lorenz-63 adding [PITH_FULL_IMAGE:figures/full_fig_p022_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

94 extracted references · 6 canonical work pages

  1. [1]

    What learning algorithm is in-context learning? Investigations with linear models

    Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? Investigations with linear models. InProceedings of the 11th International Conference on Learning Representations (ICLR), 2023. URL https:// openreview.net/forum?id=0g0X4H8yN4I

    2023

  2. [2]

    Alligood, Tim D

    Kathleen T. Alligood, Tim D. Sauer, and James A. Yorke.Chaos: An Introduction to Dynamical Systems. Textbooks in Mathematical Sciences. Springer, 1996. ISBN 978-0-387-94677-1 978-0-387-22492-3. doi: 10.1007/b97589

    doi:10.1007/b97589 1996

  3. [3]

    Dynode: Neural or- dinary differential equations for dynamics modeling in continuous control.arXiv preprint arXiv:2009.04278, 2020

    Victor M Martinez Alvarez, Rare¸ s Ro¸ sca, and Cristian G F˘alcu¸ tescu. Dynode: Neural or- dinary differential equations for dynamics modeling in continuous control.arXiv preprint arXiv:2009.04278, 2020

    Pith/arXiv arXiv 2009

  4. [4]

    Maddix, Hao Wang, Michael W

    Abdul Fatir Ansari, Lorenzo Stella, Ali Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shub- ham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, and Bernie Wang. Chronos: Learning the lan...

    2024

  5. [5]

    Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erickson, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, and Michael Bohlke-Schneider

    Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C. Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erickson, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, and Michael B...

    Pith/arXiv arXiv 2025

  6. [6]

    Benjamin Erichson, Vanessa Lin, and Michael W

    Omri Azencot, N. Benjamin Erichson, Vanessa Lin, and Michael W. Mahoney. Forecasting Se- quential Data using Consistent Koopman Autoencoders. InProceedings of the 37th International Conference on Machine Learning, 2020. URLhttp://arxiv.org/abs/2003.02236

    Pith/arXiv arXiv 2020

  7. [7]

    Transformers as statisti- cians: Provable in-context learning with in-context algorithm selection

    Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, and Song Mei. Transformers as statisti- cians: Provable in-context learning with in-context algorithm selection. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors,Advances in Neu- ral Information Processing Systems, volume 36, pages 57125–57211. Curran Associates, Inc., 2023. URL ...

    2023

  8. [8]

    xlstm: Extended long short-term memory.Advances in Neural Information Processing Systems, 37:107547– 107603, 2024

    Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xlstm: Extended long short-term memory.Advances in Neural Information Processing Systems, 37:107547– 107603, 2024

    2024

  9. [9]

    Lyapunov characteristic exponents for smooth dynamical systems and for hamiltonian systems; a method for computing all of them

    Giancarlo Benettin, Luigi Galgani, Antonio Giorgilli, and Jean-Marie Strelcyn. Lyapunov characteristic exponents for smooth dynamical systems and for hamiltonian systems; a method for computing all of them. part 1: Theory.Meccanica, 15(1):9–20, 1980

    1980

  10. [10]

    Bhalla and Ravi Iyengar

    Upinder S. Bhalla and Ravi Iyengar. Emergent properties of networks of biological signaling pathways.Science, 283(5400):381–387, 1999. doi: 10.1126/science.283.5400.381

    doi:10.1126/science.283.5400.381 1999

  11. [11]

    Mamba4cast: Efficient zero-shot time series forecasting with state space models.arXiv preprint arXiv:2410.09385, 2024

    Sathya Kamesh Bhethanabhotla, Omar Swelam, Julien Siems, David Salinas, and Frank Hutter. Mamba4cast: Efficient zero-shot time series forecasting with state space models.arXiv preprint arXiv:2410.09385, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Continuous-time piecewise-linear recurrent neural networks.arXiv preprint arXiv:2602.15649, 2026

    Alena Brändle, Lukas Eisenmann, Florian Götz, and Daniel Durstewitz. Continuous-time piecewise-linear recurrent neural networks.arXiv preprint arXiv:2602.15649, 2026. 10

    arXiv 2026

  13. [13]

    Mikhaeil, Leonard F

    Manuel Brenner, Florian Hess, Jonas M. Mikhaeil, Leonard F. Bereska, Zahra Monfared, Po- Chen Kuo, and Daniel Durstewitz. Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems. InProceedings of the 39th International Conference on Machine Learn- ing, pages 2292–2320. PMLR, June 2022. URL https://proceedings.mlr.press/v162/ brenner22a.htm...

    2022

  14. [14]

    Almost- linear rnns yield highly interpretable symbolic codes in dynamical systems reconstruction

    Manuel Brenner, Christoph Jürgen Hemmer, Zahra Monfared, and Daniel Durstewitz. Almost- linear rnns yield highly interpretable symbolic codes in dynamical systems reconstruction. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 36829–36868. Cu...

    2024

  15. [15]

    Learning interpretable hierarchical dynamical systems models from time series data

    Manuel Brenner, Elias Weber, Georgia Koppe, and Daniel Durstewitz. Learning interpretable hierarchical dynamical systems models from time series data. InThe Thirteenth International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/ forum?id=Vp2OAxMs2s

    2025

  16. [16]

    Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  17. [17]

    Cambridge University Press, 2019

    Steven L Brunton and J Nathan Kutz.Data-driven science and engineering: Machine learning, dynamical systems, and control. Cambridge University Press, 2019

    2019

  18. [18]

    Brunton, Joshua L

    Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems.Proceedings of the National Academy of Sciences USA, 113(15):3932–3937, 2016. ISSN 0027-8424. doi: 10.1073/pnas. 1517384113

    doi:10.1073/pnas 2016

  19. [19]

    Brunton, Bingni W

    Steven L. Brunton, Bingni W. Brunton, Joshua L. Proctor, Eurika Kaiser, and J. Nathan Kutz. Chaos as an intermittently forced linear system.Nature Communications, 8(1):19, May 2017. ISSN 2041-1723. doi: 10.1038/s41467-017-00030-8. URL https://www.nature.com/ articles/s41467-017-00030-8. Publisher: Nature Publishing Group

    doi:10.1038/s41467-017-00030-8 2017

  20. [20]

    Brunton, Marko Budiši´c, Eurika Kaiser, and J

    Steven L. Brunton, Marko Budiši´c, Eurika Kaiser, and J. Nathan Kutz. Modern koopman theory for dynamical systems.SIAM Review, 64(2):229–340, 2022. doi: 10.1137/21M1401243. URL https://doi.org/10.1137/21M1401243

    doi:10.1137/21m1401243 2022

  21. [21]

    Oxford University Press, August 2006

    Gyorgy Buzsaki.Rhythms of the Brain. Oxford University Press, August 2006. ISBN 978-0- 19-804125-2. Google-Books-ID: ldz58irprjYC

    2006

  22. [22]

    Carroll and Louis M

    Thomas L. Carroll and Louis M. Pecora. Network Structure Effects in Reservoir Computers. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(8):083130, August 2019. ISSN 1054-1500, 1089-7682. doi: 10.1063/1.5097686

    work page doi:10.1063/1.5097686 2019

  23. [23]

    Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural Ordinary Differential Equations. InAdvances in Neural Information Processing Systems 31, 2018. URL http://arxiv.org/abs/1806.07366

    Pith/arXiv arXiv 2018

  24. [24]

    Unveiling induction heads: Prov- able training dynamics and feature learning in transformers.Advances in Neural Information Processing Systems, 37:66479–66567, 2024

    Siyu Chen, Heejune Sheen, Tianhao Wang, and Zhuoran Yang. Unveiling induction heads: Prov- able training dynamics and feature learning in transformers.Advances in Neural Information Processing Systems, 37:66479–66567, 2024

    2024

  25. [25]

    Porter, and Yi Ming Lai

    Stephen Coombes, Mustafa ¸ Sayli, R"udiger Thul, Rachel Nicks, Mason A. Porter, and Yi Ming Lai. Oscillatory networks: Insights from piecewise-linear modeling.SIAM Review, 66(4): 619–679, 2024. doi: 10.1137/22M1534365

    doi:10.1137/22m1534365 2024

  26. [26]

    Daafouz, P

    J. Daafouz, P. Riedinger, and C. Iung. Stability analysis and control synthesis for switched systems: a switched Lyapunov function approach.IEEE Transactions on Automatic Control, 47 (11):1883–1887, November 2002. ISSN 1558-2523. doi: 10.1109/TAC.2002.804474. URL https://ieeexplore.ieee.org/document/1047016. Conference Name: IEEE Transac- tions on Automat...

    arXiv 2002

  27. [27]

    A decoder-only foundation model for time-series forecasting

    Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. InForty-first International Conference on Machine Learning, 2024

    2024

  28. [28]

    Budd, Alan R

    Mario di Bernardo, Chris J. Budd, Alan R. Champneys, and Piotr Kowalczyk.Piecewise-Smooth Dynamical Systems: Theory and Applications, volume 163 ofApplied Mathematical Sciences. Springer, London, 2008. ISBN 978-1-84628-039-9. doi: 10.1007/978-1-84628-708-4. 11

    doi:10.1007/978-1-84628-708-4 2008

  29. [29]

    A state space approach for piecewise-linear recurrent neural networks for identifying computational dynamics from neural measurements.PLoS Comput

    Daniel Durstewitz. A state space approach for piecewise-linear recurrent neural networks for identifying computational dynamics from neural measurements.PLoS Comput. Biol., 13(6): e1005542, 2017. ISSN 1553-7358. doi: 10.1371/journal.pcbi.1005542

    doi:10.1371/journal.pcbi.1005542 2017

  30. [30]

    Reconstructing computational system dynamics from neural data with recurrent neural networks.Nature Reviews

    Daniel Durstewitz, Georgia Koppe, and Max Ingo Thurm. Reconstructing computational system dynamics from neural data with recurrent neural networks.Nature Reviews. Neuroscience, 24 (11):693–710, November 2023. ISSN 1471-0048. doi: 10.1038/s41583-023-00740-7

    doi:10.1038/s41583-023-00740-7 2023

  31. [31]

    Position: Why a dynamical systems perspective is needed to advance time series modeling.arXiv preprint arXiv:2602.16864, 2026

    Daniel Durstewitz, Christoph Jürgen Hemmer, Florian Hess, Charlotte Ricarda Doll, and Lukas Eisenmann. Position: Why a dynamical systems perspective is needed to advance time series modeling.arXiv preprint arXiv:2602.16864, 2026

    Pith/arXiv arXiv 2026

  32. [32]

    The evolution of statistical induction heads: In-context learning markov chains.Advances in neural information processing systems, 37:64273–64311, 2024

    Ezra Edelman, Nikolaos Tsilivis, Benjamin L Edelman, Eran Malach, and Surbhi Goel. The evolution of statistical induction heads: In-context learning markov chains.Advances in neural information processing systems, 37:64273–64311, 2024

    2024

  33. [33]

    Doyne Farmer and John J

    J. Doyne Farmer and John J. Sidorowich. Predicting chaotic time series.Physical Review Letters, 59(8):845–848, August 1987. doi: 10.1103/PhysRevLett.59.845. URL https://link. aps.org/doi/10.1103/PhysRevLett.59.845. Publisher: American Physical Society

    doi:10.1103/physrevlett.59.845 1987

  34. [34]

    An algorithm for finding best matches in logarithmic expected time.ACM Transactions on Mathematical Software (TOMS), 3 (3):209–226, 1977

    Jerome H Friedman, Jon Louis Bentley, and Raphael Ari Finkel. An algorithm for finding best matches in logarithmic expected time.ACM Transactions on Mathematical Software (TOMS), 3 (3):209–226, 1977

    1977

  35. [35]

    Fumagalli, K

    L. Fumagalli, K. Lüdge, J. de Wiljes, H. Haario, and L. Jaurigue. Data-driven performance measures using global properties of attractors for testing black-box surrogate models of chaotic systems.Chaos: An Interdisciplinary Journal of Nonlinear Science, 35(11):113121, 11 2025. ISSN 1054-1500. doi: 10.1063/5.0283424. URLhttps://doi.org/10.1063/5.0283424

    work page doi:10.1063/5.0283424 2025

  36. [36]

    What can transformers learn in-context? a case study of simple function classes.Advances in Neural Information Processing Systems, 35:30583–30598, 2022

    Shivam Garg, Dimitris Tsipras, Percy S Liang, and Gregory Valiant. What can transformers learn in-context? a case study of simple function classes.Advances in Neural Information Processing Systems, 35:30583–30598, 2022

    2022

  37. [37]

    Chaos as an interpretable benchmark for forecasting and data-driven modelling

    William Gilpin. Chaos as an interpretable benchmark for forecasting and data-driven modelling. InThirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2022. URLhttps://openreview.net/forum?id=enYjtbjYJrf

    2022

  38. [38]

    Model scale versus domain knowledge in statistical forecasting of chaotic systems.Physical Review Research, 5(4):043252, 2023

    William Gilpin. Model scale versus domain knowledge in statistical forecasting of chaotic systems.Physical Review Research, 5(4):043252, 2023

    2023

  39. [39]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First conference on language modeling, 2024

    2024

  40. [40]

    Springer, New York, NY , 1983

    John Guckenheimer and Philip Holmes.Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, volume 42 ofApplied Mathematical Sciences. Springer, New York, NY , 1983. ISBN 978-1-4612-7020-1 978-1-4612-1140-2. doi: 10.1007/978-1-4612-1140-2. URLhttp://link.springer.com/10.1007/978-1-4612-1140-2

    doi:10.1007/978-1-4612-1140-2 1983

  41. [41]

    Out-of-Domain Generalization in Dynamical Systems Reconstruction

    Niclas Alexander Göring, Florian Hess, Manuel Brenner, Zahra Monfared, and Daniel Durste- witz. Out-of-Domain Generalization in Dynamical Systems Reconstruction. InProceedings of the 41st International Conference on Machine Learning, pages 16071–16114. PMLR, July 2024. URLhttps://proceedings.mlr.press/v235/goring24a.html. ISSN: 2640-3498

    2024

  42. [42]

    A tighter generalization bound for reservoir computing

    Xinyu Han, Yi Zhao, and Michael Small. A tighter generalization bound for reservoir computing. Chaos: An Interdisciplinary Journal of Nonlinear Science, 32(4):043115, April 2022. ISSN 1054-1500. doi: 10.1063/5.0082258

    work page doi:10.1063/5.0082258 2022

  43. [43]

    True zero-shot inference of dynamical sys- tems preserving long-term statistics

    Christoph Jürgen Hemmer and Daniel Durstewitz. True zero-shot inference of dynamical sys- tems preserving long-term statistics. InThe Thirty-ninth Annual Conference on Neural Informa- tion Processing Systems, 2025. URLhttps://openreview.net/forum?id=RE97LT26w8

    2025

  44. [44]

    Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction

    Christoph Jürgen Hemmer, Manuel Brenner, Florian Hess, and Daniel Durstewitz. Optimal Recurrent Network Topologies for Dynamical Systems Reconstruction. InProceedings of the 41st International Conference on Machine Learning, pages 18174–18204. PMLR, July 2024. URLhttps://proceedings.mlr.press/v235/hemmer24a.html. ISSN: 2640-3498

    2024

  45. [45]

    Generalized Teacher Forcing for Learning Chaotic Dynamics

    Florian Hess, Zahra Monfared, Manuel Brenner, and Daniel Durstewitz. Generalized Teacher Forcing for Learning Chaotic Dynamics. InProceedings of the 40th International Conference on Machine Learning, pages 13017–13049. PMLR, July 2023. URL https://proceedings. mlr.press/v202/hess23a.html. ISSN: 2640-3498. 12

    2023

  46. [46]

    Computer Science, Artificial Intel- ligence

    Hansika Hewamalage, Klaus Ackermann, and Christoph Bergmeir. Forecast evaluation for data scientists: common pitfalls and best practices.Data Mining and Knowledge Discov- ery, 37(2):788–832, 2023. URL https://link.springer.com/content/pdf/10.1007/ s10618-022-00894-5.pdf?pdf=button . (IF 4.8, Q2 in “Computer Science, Artificial Intel- ligence”, JCR 2022)

    2023

  47. [47]

    Training neural operators to preserve invariant measures of chaotic attractors.Advances in Neural Information Processing Systems, 36:27645–27669, 2023

    Ruoxi Jiang, Peter Y Lu, Elena Orlova, and Rebecca Willett. Training neural operators to preserve invariant measures of chaotic attractors.Advances in Neural Information Processing Systems, 36:27645–27669, 2023

    2023

  48. [48]

    Less is more: Recursive reasoning with tiny networks, 2025

    Alexia Jolicoeur-Martineau. Less is more: Recursive reasoning with tiny networks, 2025. URL https://arxiv.org/abs/2510.04871

    Pith/arXiv arXiv 2025

  49. [49]

    Cambridge university press, 2004

    Holger Kantz and Thomas Schreiber.Nonlinear time series analysis, volume 7. Cambridge university press, 2004

    2004

  50. [50]

    Generalizing to new physical systems via context-informed dynamics model

    Matthieu Kirchmeyer, Yuan Yin, Jérémie Donà, Nicolas Baskiotis, Alain Rakotomamonjy, and Patrick Gallinari. Generalizing to new physical systems via context-informed dynamics model. InInternational Conference on Machine Learning, pages 11283–11301. PMLR, 2022

    2022

  51. [51]

    Homotopy-based training of neuralodes for accurate dynamics discovery.Advances in Neural Information Processing Systems, 36:64725–64752, 2023

    Joon-Hyuk Ko, Hankyul Koh, Nojun Park, and Wonho Jhe. Homotopy-based training of neuralodes for accurate dynamics discovery.Advances in Neural Information Processing Systems, 36:64725–64752, 2023

    2023

  52. [52]

    Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI

    Georgia Koppe, Hazem Toutounji, Peter Kirsch, Stefanie Lis, and Daniel Durstewitz. Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI. PLOS Computational Biology, 15(8):e1007263, 2019. ISSN 1553-7358. doi: 10.1371/journal. pcbi.1007263

    doi:10.1371/journal 2019

  53. [53]

    Panda: A pretrained forecast model for universal representation of chaotic dynamics.arXiv preprint arXiv:2505.13755, 2025

    Jeffrey Lai, Anthony Bao, and William Gilpin. Panda: A pretrained forecast model for universal representation of chaotic dynamics.arXiv preprint arXiv:2505.13755, 2025

    arXiv 2025

  54. [54]

    Benna, and Matthew G

    Ji-An Li, Marcus K. Benna, and Matthew G. Mattar. Discovering cognitive strategies with tiny recurrent neural networks.Nature, 644:993–1001, 2025. doi: 10.1038/s41586-025-09142-4

    work page doi:10.1038/s41586-025-09142-4 2025

  55. [55]

    Dual operating modes of in-context learning

    Ziqian Lin and Kangwook Lee. Dual operating modes of in-context learning. InForty-first International Conference on Machine Learning, 2024

    2024

  56. [56]

    Jean-Christophe Loiseau and Steven L. Brunton. Constrained sparse Galerkin regression. Journal of Fluid Mechanics, 838:42–67, March 2018. ISSN 0022-1120, 1469-7645. doi: 10.1017/jfm.2017.823. Publisher: Cambridge University Press

    doi:10.1017/jfm.2017.823 2018

  57. [57]

    Deterministic nonperiodic flow.Journal of atmospheric sciences, 20(2): 130–141, 1963

    Edward N Lorenz. Deterministic nonperiodic flow.Journal of atmospheric sciences, 20(2): 130–141, 1963

    1963

  58. [58]

    Nathan Kutz, and Steven L

    Bethany Lusch, J. Nathan Kutz, and Steven L. Brunton. Deep learning for universal linear embeddings of nonlinear dynamics.Nat Commun, 9(1):4950, December 2018. ISSN 2041-1723. doi: 10.1038/s41467-018-07210-0. URL http://arxiv.org/abs/1712.09707. arXiv: 1712.09707

    Pith/arXiv arXiv 2018

  59. [59]

    Basic books, 2007

    Benoit Mandelbrot and Richard L Hudson.The Misbehavior of Markets: A fractal view of financial turbulence. Basic books, 2007

    2007

  60. [60]

    On the difficulty of learning chaotic dynamics with RNNs.Advances in Neural Information Processing Systems, 35:11297–11312, December 2022

    Jonas Mikhaeil, Zahra Monfared, and Daniel Durstewitz. On the difficulty of learning chaotic dynamics with RNNs.Advances in Neural Information Processing Systems, 35:11297–11312, December 2022

    2022

  61. [61]

    Transformation of ReLU-based recurrent neural networks from discrete-time to continuous-time

    Zahra Monfared and Daniel Durstewitz. Transformation of ReLU-based recurrent neural networks from discrete-time to continuous-time. InProceedings of the 37th International Conference on Machine Learning, 2020. URL http://proceedings.mlr.press/v119/ monfared20a.html

    2020

  62. [62]

    Mumby, Alan Hastings, and Helen J

    Peter J. Mumby, Alan Hastings, and Helen J. Edwards. Thresholds and the resilience of caribbean coral reefs.Nature, 450(7166):98–101, 2007. doi: 10.1038/nature06252. URL https://doi.org/10.1038/nature06252

    work page doi:10.1038/nature06252 2007

  63. [63]

    A Koopman Approach to Understanding Sequence Neural Models.arXiv:2102.07824 [cs, math], October 2021

    Ilan Naiman and Omri Azencot. A Koopman Approach to Understanding Sequence Neural Models.arXiv:2102.07824 [cs, math], October 2021. URL http://arxiv.org/abs/2102. 07824. arXiv: 2102.07824. 13

    Pith/arXiv arXiv 2021

  64. [64]

    In-context learning and induction heads.arXiv preprint arXiv:2209.11895, 2022

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads.arXiv preprint arXiv:2209.11895, 2022

    Pith/arXiv arXiv 2022

  65. [65]

    Otto and Clarence W

    Samuel E. Otto and Clarence W. Rowley. Linearly recurrent autoencoder networks for learning dynamics.SIAM Journal on Applied Dynamical Systems, 18(1):558–593, 2019. doi: 10.1137/ 18M1177846. URLhttps://doi.org/10.1137/18M1177846

    doi:10.1137/18m1177846 2019

  66. [66]

    Inferring stochastic low-rank recurrent neural networks from neural data

    Matthijs Pals, A Erdem Sa˘gtekin, Felix Pei, Manuel Gloeckler, and Jakob H Macke. Inferring stochastic low-rank recurrent neural networks from neural data. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors,Advances in Neural Information Processing Systems, volume 37, pages 18225–18264. Curran Associates, Inc., 2...

    work page doi:10.52202/079017-0579 2024

  67. [67]

    Dhruvit Patel and Edward Ott. Using machine learning to anticipate tipping points and extrapo- late to post-tipping dynamics of non-stationary dynamical systems.Chaos (Woodbury, N.Y.), 33 (2):023143, February 2023. ISSN 1089-7682. doi: 10.1063/5.0131787

    doi:10.1063/5.0131787 2023

  68. [68]

    Hunt, Michelle Girvan, and Edward Ott

    Jaideep Pathak, Zhixin Lu, Brian R. Hunt, Michelle Girvan, and Edward Ott. Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data.Chaos: An Interdisciplinary Journal of Nonlinear Science, 27(12):121102, December 2017. ISSN 1054- 1500, 1089-7682. doi: 10.1063/1.5010300. URL http://arxiv.org/abs/1710.07313. arXiv: ...

    Pith/arXiv arXiv 2017

  69. [69]

    Number 7 in Texts in applied mathematics

    Lawrence Perko.Differential equations and dynamical systems. Number 7 in Texts in applied mathematics. Springer, New York, 3rd ed edition, 2001. ISBN 978-0-387-95116-4

    2001

  70. [70]

    A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics

    Jason A Platt, Stephen G Penny, Timothy A Smith, Tse-Chun Chen, and Henry DI Abarbanel. A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics. Neural Networks, 153:530–552, 2022

    2022

  71. [71]

    Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers

    Jason A Platt, Stephen G Penny, Timothy A Smith, Tse-Chun Chen, and Henry DI Abarbanel. Constraining chaos: Enforcing dynamical invariants in the training of reservoir computers. Chaos: An Interdisciplinary Journal of Nonlinear Science, 33(10), 2023

    2023

  72. [72]

    A practical method for calculating largest lyapunov exponents from small data sets.Physica D: Nonlinear Phenomena, 65(1-2): 117–134, 1993

    Michael T Rosenstein, James J Collins, and Carlo J De Luca. A practical method for calculating largest lyapunov exponents from small data sets.Physica D: Nonlinear Phenomena, 65(1-2): 117–134, 1993

    1993

  73. [73]

    Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence, 1(5):206–215, 2019

    Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence, 1(5):206–215, 2019. doi: 10.1038/s42256-019-0048-x

    doi:10.1038/s42256-019-0048-x 2019

  74. [74]

    Long expressive memory for sequence modeling

    T Konstantin Rusch, Siddhartha Mishra, N Benjamin Erichson, and Michael W Mahoney. Long expressive memory for sequence modeling. InInternational Conference on Learning Representations, 2022

    2022

  75. [75]

    Embedology.Journal of statistical Physics, 65(3):579–616, 1991

    Tim Sauer, James A Yorke, and Martin Casdagli. Embedology.Journal of statistical Physics, 65(3):579–616, 1991

    1991

  76. [76]

    Self-oscillations in glycolysis 1

    Evgeni E SEL’KOV . Self-oscillations in glycolysis 1. a simple kinetic model.European Journal of Biochemistry, 4(1):79–86, 1968

    1968

  77. [77]

    Lingfeng Shen, Aayush Mishra, and Daniel Khashabi. Position: Do pretrained transformers learn in-context by gradient descent? In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors,Proceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of...

    2024

  78. [78]

    Springer Science & Business Media, March 2006

    Zhendong Sun.Switched Linear Systems: Control and Design. Springer Science & Business Media, March 2006. ISBN 978-1-84628-131-0. Google-Books-ID: u4GArZN1bmsC

    2006

  79. [79]

    Detecting strange attractors in turbulence

    Floris Takens. Detecting strange attractors in turbulence. InDynamical Systems and Turbulence, Warwick 1980, volume 898, pages 366–381. Springer, 1981. ISBN 978-3-540-11171-9 978-3- 540-38945-3. URLhttp://link.springer.com/10.1007/BFb0091924

    doi:10.1007/bfb0091924 1980

  80. [80]

    Trischler and Gabriele M.T

    Adam P. Trischler and Gabriele M.T. D’Eleuterio. Synthesis of recurrent neural networks for dynamical system simulation.Neural Networks, 80:67–78, 2016. ISSN 08936080. doi: 10. 14 1016/j.neunet.2016.04.001. URL https://linkinghub.elsevier.com/retrieve/pii/ S0893608016300314

    2016

Showing first 80 references.