Pith. sign in

REVIEW 4 major objections 5 minor 45 references

By constraining the deterministic latent dynamics of a Dreamer-style world model to a spectrally bounded Koopman backbone, Koopman Dreamer makes long-horizon imagined trajectories more stable and improves closed-loop continuous control.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 01:34 UTC pith:DRTORX2X

load-bearing objection A solid, well-evaluated spectral world-model architecture, but the title's causal claim is only partially supported; the spectral constraint matters for open-loop prediction, not clearly for closed-loop success. the 4 major comments →

arxiv 2607.19719 v2 pith:DRTORX2X submitted 2026-07-22 cs.LG cs.RO

Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imagination

Jiaqi Li , Xinglong Zhang , Haibin Xie , Yixing Lan , Wei Pan , Xin Xu This is my paper
classification cs.LG cs.RO
keywords model-based reinforcement learninglatent world modelKoopman representationspectral radiuscontinuous controllong-horizon predictionautonomous navigationbilinear control
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to show that the long-horizon instability of latent world models can be treated as a spectral problem. Instead of letting a generic recurrent network decide how latent modes persist or decay, the proposed model propagates the deterministic latent state through a Koopman-inspired linear backbone made of 2D rotation–scaling blocks with learnable, bounded radii. That gives the model an explicit knob for damping, persistence, and oscillation, so imagined trajectories neither explode nor collapse. Around the backbone, linear and low-rank bilinear action terms capture global and state-dependent control effects, and a stochastic modulation term carries local correction. A multi-step error bound separates error amplification by the spectral operator from additive stochastic and modeling residuals, explaining why an intermediate spectral radius is best. On nine proprioceptive control tasks and a simulated drone-navigation benchmark, the model improves open-loop prediction accuracy and closed-loop task completion relative to the standard world-model baseline.

Core claim

The central discovery is that explicitly parameterizing modal persistence in the deterministic latent dynamics of a world model—through a block-diagonal operator of 2×2 rotation-scaling blocks whose radii are bounded—produces substantially more stable long-horizon imagination. The operator's norm equals its spectral radius, so the learned radii directly control how fast latent perturbations grow or shrink per step. The paper couples this backbone with a linear action term, a low-rank bilinear state–action interaction, and stochastic-state modulation, and trains it with EMA teacher targets, multi-step rollout losses, and open-loop observation prediction. In proprioceptive control benchmarks t

What carries the argument

The load-bearing object is the spectrally constrained Koopman backbone A_K: a block-diagonal matrix of 2×2 rotation-scaling blocks, with the i-th block having modal radius ρ_i = ρ_min + (ρ_max − ρ_min) σ(α_i) and phase θ_i = π tanh(ω_i). Because each block is a scaled rotation, A_K is normal and its induced 2-norm equals its spectral radius, so per-step latent amplification is explicitly bounded by the largest learned radius. The controlled transition adds a linear action term B_a ā, a low-rank bilinear interaction H_θ(φ, ā) = β_b W_o^b (W_φ^b φ ⊙ W_a^b ā), and stochastic modulation B_z z, followed by element-wise clipping. The machinery's role is to make long-horizon behavior a design choic

Load-bearing premise

The theoretical bound assumes the bilinear action term is locally Lipschitz with constants that are never measured, so the paper does not verify that the effective rollout rate κ = ρ + L_b^φ is actually below one on the trained models.

What would settle it

On a trained model, estimate L_b^φ numerically (fix action, perturb latent states at representative points, take the maximum ratio of output difference to input difference) and compute κ = ρ(A_K) + L_b^φ. If κ ≥ 1 but open-loop latent error still stays flat or decreases over 64 steps, the proposed bound is not what keeps rollouts stable. Alternatively, run a contact-rich or highly discontinuous control task: if the spectrally constrained backbone's open-loop error equals or exceeds that of an unconstrained recurrent transition, the generality of the claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • World-model builders can set an explicit modal budget for latent dynamics, choosing how much damping, rotation, and near-periodic persistence imaginary trajectories should have.
  • Rollout-error analysis becomes usable at design time: if the spectral radius plus the bilinear Lipschitz constant stays below one, long-horizon imagined states stay bounded; above one, errors can grow geometrically.
  • The stability–expressiveness trade-off predicts that the optimal spectral radius is task-dependent, so practitioners should tune it against decoded observation error rather than minimizing contraction.
  • The ablation results show that state-dependent action effects are crucial in dense obstacle navigation, implying that purely linear action encodings are insufficient for many continuous-control tasks.
  • Because the stochastic prior–posterior and actor–critic machinery is retained, the spectral backbone can be combined with other reward, continuation, and policy components without changing the learning loop.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The error bound suggests a quantitative check the paper does not perform: measure L_b^φ by finite differences on a trained model and compare the predicted growth rate κ to observed long-horizon latent error; agreement would confirm the bound is the operative mechanism, mismatch would mean stability comes from clipping or optimization rather than the spectral design.
  • The strong degradation when the bilinear term is removed hints that world models for navigation need state-dependent action effects; this could generalize beyond the proposed architecture to other latent-dynamics agents.
  • The authors leave adaptive spectral ranges as future work; a natural extension is to adjust ρ_min and ρ_max during training according to the observed prediction error, making the trade-off self-tuning.
  • Because the model is evaluated only in simulation, a testable extrapolation is that the same spectral backbone will also improve long-horizon prediction on physical robots, provided the true dynamics remain within the locally Lipschitz and bounded-modal regime.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Koopman Dreamer, a Dreamer-style latent world model in which the deterministic state is propagated by a spectrally constrained linear backbone composed of 2D rotation–scaling blocks with bounded radii, augmented by linear and low-rank bilinear action terms and stochastic-state modulation. Training combines posterior-conditioned EMA teacher targets, one-step consistency, multi-step teacher and prior rollouts, and open-loop observation prediction. The authors derive a contraction-style bound on multi-step rollout error and report experiments on nine DMC proprioceptive tasks and a UAV-LiDAR navigation benchmark, including open-loop prediction, spectral-radius sweeps, and structural ablations.

Significance. If the empirical claims hold, the paper demonstrates a useful architectural inductive bias for long-horizon latent imagination in model-based RL. The strengths are real: the spectral parameterization is simple and interpretable; the open-loop evaluations are extensive and include decoded observation error rather than only latent MSE; the UAV experiments are more demanding than standard DMC benchmarks; and the authors are careful to state that the spectral constraint is not a global stability guarantee. The paper also ships a clean sensitivity analysis of the spectral-radius trade-off. Its main weakness is attribution: the headline closed-loop advantage is not robustly isolated to the spectral constraint, because the system changes several mechanisms at once and the sole spectral-only ablation shows statistically indistinguishable success. The theoretical bound is a standard contraction inequality whose key Lipschitz constants are not estimated. These issues are fixable but require additional experiments or a reframing of the central claim.

major comments (4)
  1. [Section VI-D, Table XI] The only ablation that isolates the spectral constraint ('w/o spectral') gives 72.5% success (58/80) versus 71.2% (57/80) for the full model, and the mean-return 95% CIs overlap (163.82 [140.85,184.88] vs 130.33 [107.80,151.85]). The reported difference in action variation is not accompanied by a significance test. This does not support the paper's claim that the spectrally constrained Koopman backbone specifically improves closed-loop control. To make the title-level claim load-bearing, either provide a statistical test demonstrating a robust difference, or reframe the contribution as the full system and add a control that mounts the same auxiliary losses (Eqs. 7–10) and bilinear term on a non-spectral DreamerV3 transition. As it stands, the DreamerV3 comparison changes multiple mechanisms simultaneously, so the causal role of the spectral constraint remains underdetermined.
  2. [Tables I-II and Fig. 3] No seed-level variability is reported for Koopman Dreamer, and the baseline values are taken from public JSON curves without rerunning them. Several reported advantages are modest (e.g., Reacher 979.0 vs 961.8, Walker Stand 982.4 vs 957.2, Walker Walk 942.3 vs 881.0). Without per-seed scores, standard deviations, or confidence intervals, the claims 'best on six tasks' and 'beats DreamerV3 on eight' cannot be distinguished from seed noise. Please add seed-level results or at least standard errors/CIs for all nine DMC tasks, and for the open-loop MSE aggregates in Table II.
  3. [Section V.A, Eqs. (16)–(19)] The proof of Proposition 1 assumes the bilinear term is locally Lipschitz with constants L_b^phi and L_b^a, but the authors never estimate these constants or verify that kappa_star = rho_star + L_b^phi < 1 in the trained models. Since the bound's useful regime (attenuation) requires kappa_star < 1, the proposition is a generic contraction inequality rather than a quantitative analysis of the proposed model. The spectral-radius sweep is interpreted post hoc and is not predicted by the bound. Please estimate L_b^phi from checkpoints and report kappa_star, or explicitly demote the theory to a structural motivation rather than a predictive analysis.
  4. [Section VI.B] The main UAV closed-loop comparison uses one fixed checkpoint per method and 240 episodes aggregated across scenes, with no repeated training runs. With a single seed per method, the reported success-rate gap (73.8% vs 53.8%) could reflect run-to-run variation. Please add at least 3–5 seeds with CI/standard error, or clearly label the result as a single-run demonstration. The same concern applies to the structural ablation section, which uses 80 episodes from one shared scene for each variant.
minor comments (5)
  1. [Throughout] The text consistently misspells 'UAV' as 'UA V' (e.g., 'UA V-LiDAR'). Please fix this globally.
  2. [Section IV.F] The sentence 'F . Posterior-Conditioned Teacher Representation' contains a spurious space after the section letter; this is a formatting error.
  3. [Table II] The reward MSE at H=64 is worse for Koopman Dreamer (1.2588 vs 1.1920), even though the mean-over-horizon reward MSE is better. This inconsistency should be mentioned in the text or explained, since it qualifies the otherwise favorable open-loop picture.
  4. [Section VI.A] The DMC baseline curves are taken from the public DreamerV3 benchmark rather than retrained under the same code base. Please state explicitly in the main text (not only in the appendix) that this may introduce protocol differences, and report which exact checkpoint/JSON file was used so the comparison is reproducible.
  5. [Section VI.D / Table V] The 'w/o spectral' variant removes the bounded-radius parameterization but keeps the same EMA teacher, rollout losses, and open-loop prediction loss. This is a reasonable ablation for the spectral constraint, but the text should more prominently acknowledge that it does not reduce the model to DreamerV3. The current wording in the discussion ('the auxiliary mechanisms strengthen, rather than replace, the spectral dynamics core') is stronger than the evidence supports.

Circularity Check

0 steps flagged

No circularity found: the theoretical bound is a conditional lemma, the teacher targets are observation-anchored, and the central comparisons use external baselines.

full rationale

The paper's derivation chain is self-contained. Proposition 1 (Eqs. 13-19) is a conditional lemma: given the 2-norm equality ||A_K||_2 = rho(A_K) from the block-diagonal rotation-scaling construction, the local Lipschitz assumption on H_theta, and the 1-Lipschitz property of clipping, the bound follows by triangle inequality and repeated substitution. It is not fitted to the experimental curves, and the paper explicitly labels it a local rollout-error result rather than a global stability guarantee. The teacher targets in Eq. (6) are EMA projections of the model's own posterior states, but they are anchored by observation encodings e_t and are additionally supervised against actual future observations through the open-loop prediction loss L_pred (Eq. 10), so the training signal is not purely self-referential. The central empirical comparison is against the external DreamerV3 public benchmark and a separately implemented D4PG baseline, so the improvements are not produced by construction. The only in-family citations ([32], [33]) are related-work mentions of prior Koopman control papers by overlapping authors; they are not used to justify the spectral constraint, the theoretical bound, or the empirical conclusions. The known attribution weakness—that the full-system comparison changes several components at once—is a confounding/identifiability concern about experimental design, not circularity. The acknowledged limitation that the Lipschitz constants in Eq. (16) are not estimated makes the bound potentially vacuous, but vacuousness is a weakness of quantitative sharpness, not a reduction of the result to its inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The paper's contribution includes a theorem whose applicability depends on unmeasured constants; the main architectural choices are controlled by a set of hand-set hyperparameters.

free parameters (5)
  • Spectral radius bounds [rho_min, rho_max] = DMC: [0.85, 0.95]; UAV: [0.75, 0.95]
    Hand-selected hyperparameters limiting the learned modal radii; the paper shows performance is sensitive to rho_max (Section VI-C).
  • Bilinear scale beta_b = 0.05 (UAV); not reported for DMC
    Controls the magnitude of the state-action interaction term (Eq. 3).
  • Teacher EMA rate tau_ema = 0.01
    Controls how slowly the teacher projection is updated; standard EMA hyperparameter.
  • Loss weights and warmup schedules = e.g., koop 0.02/0.005, roll 0.02/0.005, pred 0.05/0.01 (DMC/UAV)
    Balances the auxiliary losses; chosen without a systematic sweep.
  • Learned spectral radius rho(A_K) = 0.881 (UAV main run); range 0.849-1.026 in sweep
    Result of training, but it is effectively an emergent fitted parameter that the analysis treats as adjustable.
axioms (4)
  • ad hoc to paper Bilinear term H_theta is locally Lipschitz with constants L_b^phi, L_b^a (Eq. 16)
    Assumed in Proposition 1 proof; constants are not measured.
  • standard math clip_cphi(x) = cphi tanh(x/cphi) is 1-Lipschitz
    Standard property of tanh and scaling.
  • standard math A_K is a block-diagonal normal operator, so ||A_K||_2 = rho(A_K)
    Rotation-scaling blocks are normal, and block-diagonal normal operators have norm equal to spectral radius.
  • domain assumption Posterior EMA teacher targets provide a valid supervised signal for the prior rollout
    The teacher is derived from the model's own posterior states, but combined with observation prediction it is assumed sufficient.

pith-pipeline@v1.3.0-alltime-deepseek · 24254 in / 11368 out tokens · 113467 ms · 2026-08-04T01:34:07.586667+00:00 · methodology

0 comments
read the original abstract

Latent world models improve sample efficiency in continuous control by optimizing policies over imagined latent trajectories, but common neural transitions offer limited direct control over modal persistence and error accumulation in long rollouts. We propose Koopman Dreamer, a Dreamer-style world model with a spectrally constrained deterministic latent dynamics core. Its Koopman-inspired backbone uses two-dimensional rotation--scaling blocks with bounded radii to represent damping, rotation, and near-periodic modes. Linear and low-rank bilinear action terms capture global and state-dependent control effects, while stochastic-state modulation supplies local correction information. To reduce the mismatch between posterior-conditioned training and prior-only imagination, the model combines posterior-conditioned EMA teacher targets with one-step consistency, multi-step rollout, and open-loop observation-prediction objectives. We further derive a multi-step rollout-error bound that separates amplification by the spectral backbone and bilinear interaction from the additive effects of stochastic-state mismatch and modeling residuals, clarifying the trade-off between error attenuation and long-term information retention. Experimental results on proprioceptive continuous-control tasks from the DeepMind Control Suite and UAV-LiDAR autonomous navigation demonstrate that Koopman Dreamer improves the stability of long-horizon latent rollouts and achieves stronger closed-loop control performance on tasks that rely on high-quality multi-step imagination.

Figures

Figures reproduced from arXiv: 2607.19719 by Haibin Xie, Jiaqi Li, Wei Pan, Xinglong Zhang, Xin Xu, Yixing Lan.

Figure 1
Figure 1. Figure 1: Learning workflows of Koopman Dreamer. (a) Posterior-conditioned world-model learning from observed sequences, where observations provide [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 1
Figure 1. Figure 1: Learning workflows of Koopman Dreamer. (a) Posterior-conditioned world-model learning from observed sequences, where observations provide [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Controlled Koopman latent transition and its spectral structure. (a) Bounded modal radii and phases define the two-dimensional rotation–scaling blocks [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Learning curves on nine DMC proprioceptive tasks. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Environment steps required to reach 90% of the task-best final return. NR denotes that the target was not reached within 500K steps. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Top-down trajectories for four Forest targets: (a) [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 5
Figure 5. Figure 5: Top-down trajectories for four Forest targets: (a) [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Closed-loop UAV-LiDAR results over four targets and three shared Forest scenes. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Open-loop prediction MSE versus horizon in the UAV-LiDAR Forest scenario. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Horizon-64 prediction MSE as a function of the learned spectral radius. Error bars show variation across evaluation batches. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 8
Figure 8. Figure 8: Horizon-64 prediction MSE as a function of the learned spectral radius. Error bars show variation across evaluation batches. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Closed-loop ablation results: return distribution, outcome composition, and action variation on successful episodes. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Open-loop prediction errors of the ablation models versus horizon. All variants use a 32-step context and 64-step prediction horizon. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Open-loop prediction error as a function of prediction horizon under different learned spectral radii in the UAV-LiDAR Forest scenario. The six [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Complete open-loop prediction curves on DMC proprioceptive tasks, part 1. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Complete open-loop prediction curves on DMC proprioceptive tasks, part 2. [PITH_FULL_IMAGE:figures/full_fig_p020_13.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 6 linked inside Pith

  1. [1]

    R. S. Sutton and A. G. Barto,Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018

    2018

  2. [2]

    Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models,

    K. Chua, R. Calandra, R. McAllister, and S. Levine, “Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models,” inProc. Advances in Neural Information Processing Systems, 2018

    2018

  3. [3]

    When to trust your model: Model-based policy optimization,

    M. Janner, J. Fu, M. Zhang, and S. Levine, “When to trust your model: Model-based policy optimization,” inProc. Advances in Neural Information Processing Systems, 2019

    2019

  4. [4]

    World models,

    D. Ha and J. Schmidhuber, “World models,” arXiv:1803.10122, 2018

    Pith/arXiv arXiv 2018

  5. [5]

    Learning latent dynamics for planning from pixels,

    D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson, “Learning latent dynamics for planning from pixels,” inProc. International Conference on Machine Learning, 2019

    2019

  6. [6]

    Dream to control: Learn- ing behaviors by latent imagination,

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learn- ing behaviors by latent imagination,” inProc. International Conference on Learning Representations, 2020

    2020

  7. [7]

    TD-MPC2: Scalable, robust world models for continuous control,

    N. Hansen, X. Wang, and H. Su, “TD-MPC2: Scalable, robust world models for continuous control,” inProc. International Conference on Learning Representations, 2024

    2024

  8. [8]

    Mastering Atari with discrete world models,

    D. Hafner, T. Lillicrap, M. Norouzi, and J. Ba, “Mastering Atari with discrete world models,” inProc. International Conference on Learning Representations, 2021

    2021

  9. [9]

    Mastering diverse control tasks through world models,

    D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap, “Mastering diverse control tasks through world models,”Nature, vol. 640, no. 8059, pp. 647–653, 2025

    2025

  10. [10]

    DeepMind Control Suite,

    Y . Tassa et al., “DeepMind Control Suite,” arXiv:1801.00690, 2018

    Pith/arXiv arXiv 2018

  11. [11]

    MuJoCo: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “MuJoCo: A physics engine for model-based control,” inProc. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 5026–5033

    2012

  12. [12]

    Hamiltonian systems and transformation in Hilbert space,

    B. O. Koopman, “Hamiltonian systems and transformation in Hilbert space,”Proceedings of the National Academy of Sciences, vol. 17, no. 5, pp. 315–318, 1931

    1931

  13. [13]

    Dynamical systems of continuous spectra,

    B. O. Koopman and J. von Neumann, “Dynamical systems of continuous spectra,”Proceedings of the National Academy of Sciences, vol. 18, no. 3, pp. 255–263, 1932. 15

    1932

  14. [14]

    Spectral properties of dynamical systems, model reduction and decompositions,

    I. Mezi ´c, “Spectral properties of dynamical systems, model reduction and decompositions,”Nonlinear Dynamics, vol. 41, no. 1, pp. 309–325, 2005

    2005

  15. [15]

    Modern Koopman theory for dynamical systems,

    S. L. Brunton, M. Budi ˇsi´c, E. Kaiser, and J. N. Kutz, “Modern Koopman theory for dynamical systems,”SIAM Review, vol. 64, no. 2, pp. 229– 340, 2022

    2022

  16. [16]

    A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition,

    M. O. Williams, I. G. Kevrekidis, and C. W. Rowley, “A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition,”Journal of Nonlinear Science, vol. 25, no. 6, pp. 1307– 1346, 2015

    2015

  17. [17]

    Deep learning for universal linear embeddings of nonlinear dynamics,

    B. Lusch, J. N. Kutz, and S. L. Brunton, “Deep learning for universal linear embeddings of nonlinear dynamics,”Nature Communications, vol. 9, no. 1, Art. no. 4950, 2018

    2018

  18. [18]

    Learning Koopman invariant subspaces for dynamic mode decomposition,

    N. Takeishi, Y . Kawahara, and T. Yairi, “Learning Koopman invariant subspaces for dynamic mode decomposition,” inProc. Advances in Neural Information Processing Systems, 2017

    2017

  19. [19]

    Air Learning: A deep reinforcement learning gym for autonomous aerial robot visual navigation,

    S. Krishnan, B. Boroujerdian, W. Fu, A. Faust, and V . J. Reddi, “Air Learning: A deep reinforcement learning gym for autonomous aerial robot visual navigation,”Machine Learning, vol. 110, no. 9, pp. 2501– 2540, 2021, doi: 10.1007/s10994-021-06006-6

    doi:10.1007/s10994-021-06006-6 2021

  20. [20]

    LiDAR-based drone navi- gation with reinforcement learning,

    P. Miera, H. Szolc, and T. Kryjak, “LiDAR-based drone navi- gation with reinforcement learning,” arXiv:2307.14313, 2023, doi: 10.48550/arXiv.2307.14313

    Pith/arXiv arXiv doi:10.48550/arxiv.2307.14313 2023

  21. [21]

    Auto-encoding variational Bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” in Proc. International Conference on Learning Representations, 2014

    2014

  22. [22]

    Spectral analysis of nonlinear flows,

    C. W. Rowley, I. Mezi ´c, S. Bagheri, P. Schlatter, and D. S. Henningson, “Spectral analysis of nonlinear flows,”Journal of Fluid Mechanics, vol. 641, pp. 115–127, 2009

    2009

  23. [23]

    Dynamic mode decomposition of numerical and experi- mental data,

    P. J. Schmid, “Dynamic mode decomposition of numerical and experi- mental data,”Journal of Fluid Mechanics, vol. 656, pp. 5–28, 2010

    2010

  24. [24]

    Data-driven discovery of Koopman eigenfunctions for control,

    E. Kaiser, J. N. Kutz, and S. L. Brunton, “Data-driven discovery of Koopman eigenfunctions for control,”Machine Learning: Science and Technology, vol. 2, no. 3, Art. no. 035023, 2021

    2021

  25. [25]

    Linear predictors for nonlinear dynamical sys- tems: Koopman operator meets model predictive control,

    M. Korda and I. Mezi ´c, “Linear predictors for nonlinear dynamical sys- tems: Koopman operator meets model predictive control,”Automatica, vol. 93, pp. 149–160, 2018

    2018

  26. [26]

    Data-driven identification of inter- pretable reduced-order models using Koopman theory,

    A. Narasingam and J. S.-I. Kwon, “Data-driven identification of inter- pretable reduced-order models using Koopman theory,”AIChE Journal, vol. 66, no. 11, Art. no. e16993, 2020

    2020

  27. [27]

    Deep Koopman operator with control for nonlinear systems,

    H. Shi and M. Q.-H. Meng, “Deep Koopman operator with control for nonlinear systems,”IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 7700–7707, 2022

    2022

  28. [28]

    Koopman Q- learning: Offline reinforcement learning via symmetries of dynamics,

    M. Weissenbacher, S. Sinha, A. Garg, and Y . Kawahara, “Koopman Q- learning: Offline reinforcement learning via symmetries of dynamics,” inProc. 39th Int. Conf. Mach. Learn. (ICML), 2022, pp. 23645–23667

    2022

  29. [29]

    Task-oriented Koopman- based control with contrastive encoder,

    X. Lyu, H. Hu, S. Siriya, Y . Pu, and M. Chen, “Task-oriented Koopman- based control with contrastive encoder,” inProc. 7th Conf. Robot Learn. (CoRL), 2023, pp. 93–105

    2023

  30. [30]

    RoboKoop: Ef- ficient control conditioned representations from visual input in robotics using Koopman operator,

    H. Kumawat, B. Chakraborty, and S. Mukhopadhyay, “RoboKoop: Ef- ficient control conditioned representations from visual input in robotics using Koopman operator,” inProc. 8th Conf. Robot Learn. (CoRL), 2025, pp. 3474–3499

    2025

  31. [31]

    Data-driven optimal control of unknown nonlinear dynamical systems using the Koopman operator,

    Z. Zeng, R. Zhou, Y . Meng, and J. Liu, “Data-driven optimal control of unknown nonlinear dynamical systems using the Koopman operator,” inProc. 7th Annu. Learn. Dyn. Control Conf. (L4DC), 2025, pp. 1127– 1139

    2025

  32. [32]

    Receding-horizon direct policy optimization with Koopman embedding for trajectory tracking of fixed- wing UA Vs,

    H. Li, X. Zhang, J. Xie, and X. Xu, “Receding-horizon direct policy optimization with Koopman embedding for trajectory tracking of fixed- wing UA Vs,”Science China Technological Sciences, vol. 69, no. 4, Art. no. 1420303, 2026, doi: 10.1007/s11431-025-3179-9

    doi:10.1007/s11431-025-3179-9 2026

  33. [33]

    Learning predictive control with deep Koopman operators for au- tonomous vehicle motion planning,

    X. Zhang, Y . Xiao, H. Cao, X. Zhou, X. Yin, and X. Xu, “Learning predictive control with deep Koopman operators for au- tonomous vehicle motion planning,” arXiv:2606.08136, 2026, doi: 10.48550/arXiv.2606.08136

    Pith/arXiv arXiv doi:10.48550/arxiv.2606.08136 2026

  34. [34]

    Unitary evolution recurrent neu- ral networks,

    M. Arjovsky, A. Shah, and Y . Bengio, “Unitary evolution recurrent neu- ral networks,” inProc. International Conference on Machine Learning, 2016

    2016

  35. [35]

    Stable recurrent models,

    J. Miller and M. Hardt, “Stable recurrent models,” inProc. International Conference on Learning Representations, 2019

    2019

  36. [36]

    Categorical reparameterization with Gumbel-Softmax,

    E. Jang, S. Gu, and B. Poole, “Categorical reparameterization with Gumbel-Softmax,” inProc. International Conference on Learning Rep- resentations, 2017

    2017

  37. [37]

    The Concrete distribution: A continuous relaxation of discrete random variables,

    C. J. Maddison, A. Mnih, and Y . W. Teh, “The Concrete distribution: A continuous relaxation of discrete random variables,” inProc. Inter- national Conference on Learning Representations, 2017

    2017

  38. [38]

    Mean teachers are better role mod- els: Weight-averaged consistency targets improve semi-supervised deep learning results,

    A. Tarvainen and H. Valpola, “Mean teachers are better role mod- els: Weight-averaged consistency targets improve semi-supervised deep learning results,” inProc. Advances in Neural Information Processing Systems, 2017

    2017

  39. [39]

    Bootstrap your own latent: A new approach to self-supervised learning,

    J.-B. Grill et al., “Bootstrap your own latent: A new approach to self-supervised learning,” inProc. Advances in Neural Information Processing Systems, 2020

    2020

  40. [40]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  41. [41]

    Continuous control with deep reinforcement learn- ing,

    T. P. Lillicrap et al., “Continuous control with deep reinforcement learn- ing,” inProc. International Conference on Learning Representations, 2016

    2016

  42. [42]

    Distributed distributional deterministic policy gradients,

    G. Barth-Maron et al., “Distributed distributional deterministic policy gradients,” inProc. International Conference on Learning Representa- tions, 2018

    2018

  43. [43]

    Maximum a posteriori policy optimisation,

    A. Abdolmaleki et al., “Maximum a posteriori policy optimisation,” in Proc. International Conference on Learning Representations, 2018

    2018

  44. [44]

    A distributional view on multi-objective policy optimization,

    A. Abdolmaleki et al., “A distributional view on multi-objective policy optimization,” inProc. International Conference on Machine Learning, 2020

    2020

  45. [45]

    Gymnasium: A standard interface for reinforcement learning environments,

    M. Towers et al., “Gymnasium: A standard interface for reinforcement learning environments,” arXiv:2407.17032, 2024. 16 TABLE VI MAINHYPERPARAMETERS OFKOOPMANDREAMER ONDMC PROPRIOCEPTIVETASKS Category Hyperparameter Value Environment and training Task set 9 DMC proprioceptive tasks, env.dmc.image=False Environment and training Interaction budget 500K envi...

    Pith/arXiv arXiv 2024