Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Reinforcement-Learning-Guided Data-Driven Estimation of Spectral Properties of Stochastic Koopman Semigroups

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes Reinforced SDMD, a data-acquisition method in which a reinforcement-learning agent chooses where trajectories start so that Stochastic Dynamic Mode Decomposition better estimates the leading Koopman eigenvalues and eigen

desk verdict RL-guided sampling for Koopman estimation is a new idea, but the paper doesn't show it actually helps: no baseline, no ground truth, and a self-consistency reward that may just keep the model comfortable. read the letter →

arxiv 2509.04265 v3 pith:2EO43TNK submitted 2025-09-04 math.DS

Yuanchao Xu , Jing Liu , Weiping Ding , Zhongwei Shen , Isao Ishikawa This is my paper · ORCID
classification math.DS
keywords Koopmanoperatorstochasticdynamicalsystemsdynamicmodedecompositionreinforcementlearningspectralconsistencyestimationmetastabilitydataacquisition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Koopman spectral analysis turns a nonlinear stochastic system into a linear evolution of observables, but the accuracy of standard estimators depends on where trajectory data happen to be collected. This paper proposes to stop treating data collection as a fixed input: a reinforcement-learning agent chooses the starting points of new trajectories, Stochastic Dynamic Mode Decomposition updates an approximation of the Koopman semigroup, and a spectral-consistency reward tells the agent how well the current eigenpairs predict evolution on the newly gathered data. On double-well, Duffing, and FitzHugh–Nagumo systems, the learned policies concentrate sampling near potential minima, basins of attraction, and slow manifolds without prior dynamical knowledge, and the leading eigenfunctions settle into the expected metastable structure. The paper also proves convergence bounds linking the final policy quality to the SDMD operator error. If correct, the framework turns passive spectral analysis into an active search: with the same data budget, practitioners can get better estimates of decay rates, oscillations, and metastable behavior.

What carries the argument

The load-bearing mechanism is the spectral-consistency reward R_t = R_0 − L_SC + α_exp / (η(x_new) + ε), where L_SC = Σ_i ||φ_i(Y) − μ_i φ_i(X)||²_ρ measures how well the SDMD-estimated eigenpairs (μ_i, φ_i) reproduce the observed evolution of new trajectory data, and η is a kernel density estimate over past starting points. This reward links the RL policy to Koopman spectral quality: it tells the agent which regions make the current eigenpairs predictive. It is balanced by the exploration bonus α_exp/η so the agent does not collapse onto a single well-sampled region. SDMD supplies the operator estimate and hence the eigenpairs entering the reward.

What would settle it

Run Reinforced SDMD on a double-well system starting from a deliberately poor SDMD model that is wrong near one well. If the learned policy keeps sampling the well the model already fits because spectral-consistency reward is high there, while final eigenvalue error stays large or worsens compared with uniform sampling, then the reward is not tracking true estimation error and the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that the quality of a data-driven Koopman spectral estimate is not fixed by the dictionary or the estimator alone; it can be improved by choosing where the trajectories start, and this choice can be learned. Reinforced SDMD formulates that choice as a reinforcement-learning problem: the action space is a partition of the state space into boxes, the state is a sliding window of recent starting points, and the reward is the spectral-consistency error of the current SDMD eigenpairs on the new data plus an exploration bonus. The paper reports that the learned Q-values and policies rediscover the double-well potential landscape, the two basins of the Duffing oscillator, and t

Load-bearing premise

The whole method depends on the assumption that rewarding the current model for predicting new data well will push the sampler toward regions that make the model more accurate—not merely toward regions where the model is already comfortable.

Editorial extensions

If this is right

  • Data collection for Koopman spectral estimation can be automated: an RL agent with no prior dynamical knowledge learns which regions carry spectral information and biases trajectory initializations there.
  • Given a fixed data budget, concentrating trajectories in reward-identified regions should reduce error in the leading Koopman eigenvalues and eigenfunctions compared with spatially uniform sampling.
  • The final estimation accuracy is bounded by the SDMD error ε_sdmd: once the sampling policy is good, the remaining bottleneck is the operator estimator itself, motivating better estimators alongside better sampling.
  • The same coupling works for stateless bandit, deep Q-learning, and policy-gradient RL, so the framework is not tied to one learning algorithm.
  • The theoretical bounds mean the sampling policy's quality directly enters the spectral estimation guarantee, justifying the RL-loop design.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension not run in the paper is to compare Reinforced SDMD against an active-learning baseline that samples where the current model's predictions are most uncertain; disagreement between the two criteria would reveal whether spectral consistency is a proxy for information gain or a separate objective.
  • The convergence analysis treats the per-region reward as stationary, but in the implemented algorithm the SDMD model is updated as data accumulate, so the reward landscape drifts; non-stationary bandit analysis or freezing the model during acquisition would tighten the match between theory and practice.
  • The learned reward/Q-map could serve as a cheap pre-sampling diagnostic: a short Reinforced SDMD run identifies dynamically important regions before a full Koopman analysis, and this information may transfer to similar systems.
  • Because the action space is a grid of boxes, cost grows with dimension; a continuous-action variant, identified by the authors as future work, would be the natural next step toward higher-dimensional systems.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Reinforced SDMD, a method that couples Stochastic Dynamic Mode Decomposition with reinforcement learning to choose trajectory initialization regions. An RL agent (multi-armed bandit, DQN, or PPO) selects spatial regions, and a reward combining spectral consistency of the current SDMD eigenpairs with an exploration bonus guides the data acquisition. The authors present qualitative experiments on a double-well potential, a stochastic Duffing oscillator, and the FitzHugh--Nagumo model, and provide a convergence analysis that propagates a bounded SDMD operator error through bandit, approximate value iteration, and approximate policy iteration bounds. The central claim is that the learned sampling policy improves estimation of the leading Koopman eigenpairs.

Significance. If the central claim were established, the paper would address an important practical problem: where to collect trajectory data for Koopman spectral estimation. The integration of RL with SDMD is original, and the pseudocode and theoretical error-propagation framework are useful starting points. The paper's theory is internally consistent under its assumptions, but the assumptions and the stated reward do not connect to the central claim. The experimental sections provide only qualitative evidence and no comparison with standard sampling strategies. The significance of the contribution therefore depends entirely on whether the reward actually improves true spectral estimation error, and this is not demonstrated.

major comments (4)
  1. [Section 4, Figures 2, 3, 5, 6] The central claim that RL-guided initialization improves estimation of leading Koopman eigenpairs is not tested quantitatively. There is no comparison against random or uniform trajectory initialization, no error metric for eigenvalues or eigenfunctions, and no error bars or multiple-seed analysis. For the FHN experiment, the analytic eigenvalues λ_n = -n^2 μ + i n ω are mentioned in §4.3 but are never compared with the numerical eigenvalues in Fig. 6; moreover those formulas are stated for the noiseless system, so their applicability to the stochastic experiment is unclear. The figures are qualitative snapshots and do not support the causal statement that the learned policy improves spectral estimates.
  2. [Section 3.2, Eqs. (3.4)-(3.5)] The spectral-consistency reward is computed with the current SDMD estimate. A low value of L_Spectral_Consistency on newly generated data only shows that the new region is consistent with the current estimate; it does not show that the estimated eigenpairs are accurate. Regions where the current model is inaccurate would produce high residuals and hence low reward, so maximizing this reward may steer sampling away from the very regions that would supply corrective data. The exploration bonus is a hand-tuned counterweight, and no analysis or experiment demonstrates that it overcomes this avoidance. Without this link, the central claim is unsupported.
  3. [Section 5.1, Assumption 5.2 and Theorem 5.3] The regret analysis is disconnected from the implemented algorithm. The true reward R_a is never defined in terms of Koopman spectral estimation error; even if a stationary reward existed, Assumption 5.2 assumes ε_sdmd < Δ_min/2, which is effectively the conclusion needed to guarantee that the reward estimates resolve the best arm. In the actual algorithm, per-arm rewards are recomputed as the SDMD model evolves and include a time-varying exploration bonus, so the arms are non-stationary and the stationary-bandit regret bound does not apply to the implemented procedure.
  4. [Sections 5.2-5.3, Theorems 5.6 and 5.8] These theorems propagate a generic SDMD operator-error bound ε_sdmd through standard approximate value iteration and approximate policy iteration arguments. They do not show that the RL-selected sampling reduces ε_sdmd, nor that the learned policy improves the leading Koopman eigenpairs. Consequently, the theoretical analysis does not establish the paper's stated claim that the final estimation accuracy is linked to the quality of the learned sampling policy.
minor comments (5)
  1. [Eq. (3.4)] The expression L_Spectral_Consistency appears to be a formatting or notation error; define a clear symbol such as L_sc and specify the norm used explicitly.
  2. [Section 4.1] The reward map is only visually compared with the potential landscape; a quantitative correlation measure would strengthen the claim that the algorithm rediscovers the potential well structure.
  3. [Appendix A.1] The source code link is missing; either include the link or remove the reproducibility statement.
  4. [Section 4.3] The analytic eigenvalues are stated for the noiseless FHN system while the experiment includes noise; explain how these formulas are used as ground truth or clarify their role.
  5. [Algorithms 2 and 3] The state is a sliding window of starting points, but the reward is computed after running SDMD on an accumulated dataset; the pseudocode does not specify how often SDMD is retrained or how the dictionary is updated over time.

Circularity Check

1 steps flagged · score 5.0 of 10

Moderate circularity: the sampling reward is computed from the very eigenpairs the method aims to improve, so the learned policy is rewarded for self-consistency rather than for true Koopman accuracy.

  1. fitted input called prediction [Abstract; Section 3.2, Eqs. (3.4)-(3.5)]
    "The learned policies place more samples in regions that are useful for estimating the leading Koopman eigenpairs. ... L_Spectral Consistency := Σ_i ∥φ_i(Y) − μ_i φ_i(X)∥^2_ρ. A lower spectral consistency error implies a better model and thus a higher reward."

    The reward that guides sampling is computed from the SDMD-estimated eigenpairs (μ_i, φ_i) themselves, which are exactly the quantities whose accuracy the method claims to improve. The RL agent is therefore trained to minimize the residual of the current model on newly selected data. No equation or theorem in the paper connects this self-consistency residual to the true Koopman eigenpair error; the 'usefulness' of a region is thus defined as agreement with the current estimate. The central claim that the policy improves estimation of the leading eigenpairs reduces to a statement about the fitted model's own loss, not an independent prediction.

full rationale

The main circularity is in the reward loop: Eq. (3.4) defines spectral consistency using the current SDMD eigenpairs, and Eq. (3.5) turns it into the RL reward. Since the agent is rewarded for maximizing this quantity, it is rewarded for selecting regions where the current model extrapolates consistently, not necessarily where the true Koopman eigenpairs are best identified. The exploration bonus is the only model-independent counterweight, but its strength is a hand-chosen constant and no analysis shows it overcomes the self-consistency bias. The convergence theorems in Section 5 are standard error-propagation results conditional on Assumption 5.1 (imported from the authors' prior work [41]) and Assumption 5.2, which assumes the reward estimator can resolve the optimal arm; they do not establish that maximizing the spectral-consistency reward reduces true Koopman spectral error. Because the paper does provide qualitative experimental validation on systems with known metastable structure, the circularity is partial rather than total, so a score of 5 is appropriate. The self-citation to [41] is load-bearing for the theoretical bounds but is used as an assumption rather than as a proof of the central claim, so it does not by itself raise the score further.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

Six named free-parameter groups and five assumptions carry the claim. None is grounded in an external benchmark: the keystone SDMD error bound comes from the authors' own prior arXiv preprint, the bandit theorem's key condition is its own conclusion restated, and the reward-accuracy link is untested. The RL algorithms contribute standard machinery; what is borrowed from the literature is textbook, and what is new is not demonstrated with ground-truth error. The framework introduces no new physical entities.

free parameters (6)
  • Exploration coefficient alpha_exp = 0.15 for the double well; values for other systems not stated
    Eq. 3.5 balances spectral consistency against the KDE exploration bonus; hand-chosen, and its scaling relative to R0 and L is not justified.
  • epsilon-greedy exploration rate = 0.35 for bandit; decaying schedule for DQN, unspecified for PPO
    Controls exploration/exploitation in all three algorithms; standard but hand-chosen.
  • Baseline reward R0 and inverse-density offset in Eq. 3.5 = not stated
    Shift and regularizer of the reward; values not reported, affecting comparability of rewards across systems.
  • SDMD dictionary ('simple neural network') = unspecified architecture and training details
    Dictionary basis Psi(x;theta) is fitted via Eq. 2.8; architecture and optimization are not described, so the fitted basis is unreproducible.
  • SDMD sampling time Delta t and dictionary size N = not stated
    The estimator in Eq. 2.7 depends on both; neither is reported for the experiments.
  • DQN/PPO hyperparameters and state window length l = replay 20,000 in App. A.3; gamma, tau given symbolically; l and architectures not stated
    The state length l, network widths, minibatch sizes, and PPO clip values are not given, so the experiments cannot be reproduced as specified.
assumptions (5)
  • standard math Assumption 2.1: the stochastic Koopman semigroup {K_t} is a C0-semigroup on L^2(M, rho)
    Section 2.1, used to guarantee a dense domain and closed generator A via Eq. 2.4. Standard functional analytic background.
  • domain assumption Assumption 5.1: SDMD error is uniformly bounded, ||K - Khat|| <= eps_sdmd, as established in [41]
    Section 5, keystone of all three theorems. The bound is imported from the authors' own arXiv:2501.13301 (overlapping authorship) and its explicit dependence on dataset size, dictionary, and sampling time is not restated.
  • ad hoc to paper Assumption 5.2: eps_sdmd < Delta_min/2, i.e., SDMD reward error is smaller than half the minimum suboptimal arm gap
    Section 5.1 and Appendix B.1. This condition is effectively the bandit theorem's conclusion; Remark 5.4 concedes linear regret when it fails, so the theorem restates the assumption.
  • domain assumption Assumptions 5.5 and 5.7: Q-functions and value/reward functions are linear in a dictionary g(s)
    Sections 5.2 and 5.3, standard linear-approximation assumptions in RL theory; needed to apply AVI and API bounds.
  • ad hoc to paper Implicit: per-arm mean reward R_a is stationary and spectral-consistency reward correlates with true estimation accuracy
    Section 3.2 and Theorem 5.3. The actual reward (Eq. 3.5) is recomputed with an evolving SDMD model and a time-varying exploration bonus, so R_a is non-stationary; the correlation between self-consistency and accuracy is the paper's central untested premise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcement-Learning-Guided Data-Driven Estimation of Spectral Properties of Stochastic Koopman Semigroups." pith.science (2026). https://pith.science/paper/2EO43TNK

@misc{pith2026250904265,
  author       = {Pith},
  title        = {Pith review of: Reinforcement-Learning-Guided Data-Driven Estimation of Spectral Properties of Stochastic Koopman Semigroups},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2EO43TNK}},
  note         = {Machine review of arXiv:2509.04265}
}
read the original abstract

Koopman spectral analysis turns nonlinear stochastic dynamics into a linear evolution of observables and gives access to decay rates, oscillatory modes, and metastable behavior. In practice, however, EDMD, SDMD, and related estimators depend strongly on where the trajectory data are collected. If most trajectories start in regions that carry little spectral information, the leading eigenvalues and eigenfunctions can be poorly estimated even with a rich dictionary. We propose \emph{Reinforced SDMD}, a data-acquisition method that couples Stochastic Dynamic Mode Decomposition with reinforcement learning. The RL agent chooses trajectory-initialization regions, SDMD updates the Koopman approximation, and a spectral-consistency reward evaluates the estimated eigenpairs on the newly generated data. An exploration bonus is added to avoid repeatedly sampling only a small part of the state space. We test multi-armed bandits, DQN, and PPO on stochastic double-well, Duffing, and FitzHugh--Nagumo systems. The learned policies place more samples in regions that are useful for estimating the leading Koopman eigenpairs. We also give an error-propagation analysis showing how SDMD operator error enters the corresponding bandit, approximate value-iteration, and approximate policy-iteration bounds.

Figures

Figures reproduced from arXiv: 2509.04265 by the authors.

Figure 2
Figure 2. 1st & 2nd Koopman eigenfunctions of double well potential system at different time steps. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Duffing Oscillator: Phase Space and 1st & 2nd Koopman Eigenfunctions [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. FitzHugh-Nagumo Model Phase Portrait The FHN system’s dynamics exhibit multi-scale behavior due to the small parameter ϵ, which separates the fast dynamics of x from the slow dynamics of y. The phase space trajectory consists of rapid transitions (fast phases) and slow drifts along the cubic nullcline (x˙ = 0). Specifically, the system moves slowly along the left and right branches of the x-nullcline (where y ≈ x − … view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: 2nd approximated Koopman eigenfunctions of FitzHugh-Nagumo model at different time steps. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Approximated Koopman eigenvalues of FitzHugh-Nagumo model at different time steps. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: 1st and 2nd eigenfunctions and eigenvalues of Koopman generator with noise [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 33 canonical work pages

  1. [1]

    Arnold.Mathematical methods of classical mechanics, volume 60

    Vladimir I. Arnold.Mathematical methods of classical mechanics, volume 60. Springer Science & Business Media, 2013

    work page 2013

  2. [2]

    Batchelor.An introduction to fluid dynamics

    George K. Batchelor.An introduction to fluid dynamics. Cambridge university press, 2000

    work page 2000

  3. [3]

    Athena scientific, 2012

    Dimitri Bertsekas.Dynamic programming and optimal control: Volume I, volume 4. Athena scientific, 2012

    work page 2012

  4. [4]

    Athena Scientific, 2008

    Dimitri Bertsekas and John N Tsitsiklis.Introduction to probability, volume 1. Athena Scientific, 2008

    work page 2008

  5. [5]

    Brunton and Nathan J

    Steven L. Brunton and Nathan J. Kutz.Data-driven science and engineering: Machine learning, dynamical systems, and control. Cambridge University Press, 2022

    work page 2022

  6. [6]

    Brunton, Joshua L

    Steven L. Brunton, Joshua L. Proctor, and Nathan J. Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems.Proceedings of the national academy of sciences, 113(15): 3932–3937, 2016

    work page 2016

  7. [7]

    Colbrook and Alex Townsend

    Matthew J. Colbrook and Alex Townsend. Rigorous data-driven computation of spectral properties of Koopman operators for dynamical systems.Communications on Pure and Applied Mathematics, 77(1):221–283, 2024

    2024

  8. [8]

    David L. Donoho. Compressed sensing.IEEE Transactions on Information Theory, 52(4):1289–1306, 2006. doi: 10.1109/TIT.2006.871582. 11

    work page arXiv 2006

Show all 42 references
  1. [9]

    Impulses and physiological states in theoretical models of nerve membrane.Biophysical journal, 1(6):445–466, 1961

    Richard FitzHugh. Impulses and physiological states in theoretical models of nerve membrane.Biophysical journal, 1(6):445–466, 1961

    1961

  2. [10]

    Springer Science & Business Media, 2013

    John Guckenheimer and Philip Holmes.Nonlinear oscillations, dynamical systems, and bifurcations of vector fields, volume 42. Springer Science & Business Media, 2013

    2013

  3. [11]

    Hale.Ordinary differential equations

    Jack K. Hale.Ordinary differential equations. Courier Corporation, 2009

    2009

  4. [12]

    Hirsch, Stephen Smale, and Robert L

    Morris W. Hirsch, Stephen Smale, and Robert L. Devaney.Differential equations, dynamical systems, and an introduction to chaos. Academic press, 2013

    2013

  5. [13]

    Cambridge university press, 2012

    Philip Holmes.Turbulence, coherent structures, dynamical systems and symmetry. Cambridge university press, 2012

    2012

  6. [14]

    Peter J. Huber. Robust estimation of a location parameter. InBreakthroughs in statistics: Methodology and distribution, pages 492–518. Springer, 1992

    1992

  7. [15]

    Koopman operators with intrinsic observables in rigged reproducing kernel hilbert spaces.arXiv preprint arXiv:2403.02524, 2024

    Isao Ishikawa, Yuka Hashimoto, Masahiro Ikeda, and Yoshinobu Kawahara. Koopman operators with intrinsic observables in rigged reproducing kernel hilbert spaces.arXiv preprint arXiv:2403.02524, 2024

    2024

  8. [16]

    Asymptotic phase and amplitude for classical and semiclassical stochastic oscillators via koopman operator theory.Mathematics, 9(18):2188, 2021

    Yuzuru Kato, Jinjie Zhu, Wataru Kurebayashi, and Hiroya Nakao. Asymptotic phase and amplitude for classical and semiclassical stochastic oscillators via koopman operator theory.Mathematics, 9(18):2188, 2021

    2021

  9. [17]

    Near-optimal reinforcement learning in polynomial time.Machine learning, 49(2):209–232, 2002

    Michael Kearns and Satinder Singh. Near-optimal reinforcement learning in polynomial time.Machine learning, 49(2):209–232, 2002

    2002

  10. [18]

    Data- driven approximation of the koopman generator: Model reduction, system identification, and control.Physica D: Nonlinear Phenomena, 406:132416, 2020

    Stefan Klus, Feliks Nüske, Sebastian Peitz, Jan-Hendrik Niemann, Cecilia Clementi, and Christof Schütte. Data- driven approximation of the koopman generator: Model reduction, system identification, and control.Physica D: Nonlinear Phenomena, 406:132416, 2020

    2020

  11. [19]

    Bernard O. Koopman. Hamiltonian systems and transformation in hilbert space.Proceedings of the National Academy of Sciences, 17(5):315, 1931

    1931

  12. [20]

    Koopman and John von Neumann

    Bernard O. Koopman and John von Neumann. Dynamical systems of continuous spectra.Proceedings of the National Academy of Sciences, 18(3):255–263, 1932. doi: 10.1073/pnas.18.3.255

    1932 doi

  13. [21]

    Thomas, and Yao Li

    Max Kreider, Peter J. Thomas, and Yao Li. Artificial neural network solver for fokker-planck and koopman eigenfunctions.arXiv preprint arXiv:2508.20339, 2025

    2025 arXiv

  14. [22]

    Kutz, Steven L

    Nathan J. Kutz, Steven L. Brunton, Bingni W. Brunton, and Joshua L. Proctor.Dynamic mode decomposition: data-driven modeling of complex systems. SIAM, 2016

    2016

  15. [23]

    Qianxiao Li, Felix Dietrich, Erik M Bollt, and Ioannis G Kevrekidis. Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the koopman operator.Chaos: An Interdisciplinary Journal of Nonlinear Science, 27(10), 2017

    2017

  16. [24]

    Spectral properties of dynamical systems, model reduction and decompositions.Nonlinear Dynamics, 41:309–325, 2005

    Igor Mezi´c. Spectral properties of dynamical systems, model reduction and decompositions.Nonlinear Dynamics, 41:309–325, 2005

    2005

  17. [25]

    Playing atari with deep reinforcement learning.arXiv preprint arXiv:1312.5602, 2013

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning.arXiv preprint arXiv:1312.5602, 2013

    2013 arXiv

  18. [26]

    Error bounds for approximate value iteration

    Rémi Munos. Error bounds for approximate value iteration. InProceedings of the National Conference on Artificial Intelligence, volume 20, page 1006. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2005

    1999

  19. [27]

    Murray.Mathematical biology: I

    James D. Murray.Mathematical biology: I. An introduction, volume 17. Springer Science & Business Media, 2007

    2007

  20. [28]

    An active pulse transmission line simulating nerve axon

    Jinichi Nagumo, Suguru Arimoto, and Shuji Yoshizawa. An active pulse transmission line simulating nerve axon. Proceedings of the IRE, 50(10):2061–2070, 1962

  21. [29]

    Pavliotis.Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations

    Grigorios A. Pavliotis.Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations. Texts in Applied Mathematics. Springer New York, 2016. ISBN 9781493954797. URL https://books.google.ca/books?id=jXAsvgAACAAJ

    2016

  22. [30]

    Applied Mathematical Sciences

    Amnon Pazy.Semigroups of Linear Operators and Applications to Partial Differential Equations. Applied Mathematical Sciences. Springer New York, 2012. ISBN 9781461255611. URL https://books.google.ca/ books?id=DQvpBwAAQBAJ

    2012

  23. [31]

    Rowley, Igor Mezi´c, Shervin Bagheri, Philipp Schlatter, and Dan S

    Clarence W. Rowley, Igor Mezi´c, Shervin Bagheri, Philipp Schlatter, and Dan S. Henningson. Spectral analysis of nonlinear flows.Journal of fluid mechanics, 641:115–127, 2009. 12

    2009

  24. [32]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  25. [33]

    Introduction to multi-armed bandits.Foundations and Trends® in Machine Learning, 12 (1-2):1–286, 2019

    Aleksandrs Slivkins. Introduction to multi-armed bandits.Foundations and Trends® in Machine Learning, 12 (1-2):1–286, 2019

    2019

  26. [34]

    Strogatz.Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering (studies in nonlinearity), volume 1

    Steven H. Strogatz.Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering (studies in nonlinearity), volume 1. Westview press, 2001

    2001

  27. [35]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto.Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998

    1998

  28. [36]

    Definition and data-driven reconstruction of asymptotic phase and amplitudes of stochastic oscillators via koopman operator theory

    Shohei Takata, Yuzuru Kato, and Hiroya Nakao. Definition and data-driven reconstruction of asymptotic phase and amplitudes of stochastic oscillators via koopman operator theory. InIUTAM symposium on Nonlinear dynamics for design of mechanical systems across different length/ti...

    2023

  29. [37]

    Tu.Dynamic mode decomposition: Theory and applications

    Jonathan H. Tu.Dynamic mode decomposition: Theory and applications. PhD thesis, Princeton University, 2013

    2013

  30. [38]

    Texts in Applied Mathematics

    Stephen Wiggins.Introduction to Applied Nonlinear Dynamical Systems and Chaos. Texts in Applied Mathematics. Springer New York, NY , 2 edition, 2003. ISBN 978-0-387-00177-7. doi: 10.1007/b97481

    2003 doi

  31. [39]

    Williams, Ioannis G

    Matthew O. Williams, Ioannis G. Kevrekidis, and Clarence W. Rowley. A data–driven approximation of the koop- man operator: Extending dynamic mode decomposition.Journal of Nonlinear Science, 25(6):1307–1346, June 2015. ISSN 1432-1467. doi: 10.1007/s00332-015-9258-5. URL http://...

    2015 doi

  32. [40]

    Williams, Clarence W

    Matthew O. Williams, Clarence W. Rowley, and Ioannis G. Kevrekidis. A kernel-based method for data-driven koopman spectral analysis.Journal of Computational Dynamics, 2(2):247–265, Dec 2015. doi: 10.3934/jcd. 2015005

    2015 doi

  33. [41]

    A data- driven framework for koopman semigroup estimation in stochastic dynamical systems, 2025

    Yuanchao Xu, Kaidi Shao, Isao Ishikawa, Yuka Hashimoto, Nikos Logothetis, and Zhongwei Shen. A data- driven framework for koopman semigroup estimation in stochastic dynamical systems, 2025. URL https: //arxiv.org/abs/2501.13301

    2025 arXiv

  34. [42]

    Reskoopnet: Learning koopman representations for complex dynamics with spectral residuals

    Yuanchao Xu, Kaidi Shao, Nikos Logothetis, and Zhongwei Shen. Reskoopnet: Learning koopman representations for complex dynamics with spectral residuals. InProceedings of the 42nd International Conference on Machine Learning (ICML). PMLR, 2025. 13 Appendix A A.1 Source Code For...

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.