Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Time-Aware World Model for Adaptive Prediction and Control

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read World models that know the time step between observations can keep control quality when sensors slow down.

desk verdict Useful empirical recipe for time-conditioned world models, but the causal gain is not yet isolated from mixture training and the theory section overclaims. read the letter →

arxiv 2506.08441 v1 pith:JA2P57RO submitted 2025-06-10 cs.LG cs.AIcs.SYeess.SY

Anh N. Nhu , Sanghyun Son , Ming Lin This is my paper · ORCID
classification cs.LGcs.AIcs.SYeess.SY
keywords time-awareworldmodelmodel-basedreinforcementlearningvariabletimestepmulti-scaledynamicssampleefficiencyNyquist-Shannonsamplinglatentpredictivecontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a world model for control should know how much time passes between observations. It introduces a time-aware model whose latent dynamics, reward, value, and policy are all conditioned on the step size $\Delta t$, and trains it with log-uniformly sampled $\Delta t$ values instead of one fixed rate. The claimed payoff is that a single model trained with the same number of samples and steps matches a fixed-rate baseline at the default 2.5 ms rate and substantially outperforms it at coarser evaluation rates up to 50 ms, on manipulation and PDE-control tasks. If right, this makes world models usable when sensors or actuators run slower than the training setup, and removes the need to retrain a separate model for every observation rate.

What carries the argument

The central object is a time-aware latent transition: an encoder maps an observation to a latent state $z_t$, and a network $d$ predicts a latent derivative that is integrated over a rescaled step $\tau(\Delta t)$ via Euler or fourth-order Runge-Kutta integration, with $\tau(\Delta t)=\max(0,\log_{10}\Delta t+5)$. Conditioning the reward, value, and policy networks on $\Delta t$ as well makes every time-dependent component aware of the observation interval. The training mechanism is mixture-of-step-size sampling: at episode start, $\Delta t$ is drawn log-uniformly over a range such as 1 to 50 ms for the main tasks, so a single buffer contains transitions at many rates and the same network must interpolate across them. The theoretical supporting device is Lemma 4.1, an interpolation identity for the optimal dynamics function, plus Lemma 4.2, which states that reducing modeling error at a large scale lowers the error bound at all smaller scales.

What would settle it

Train the time-aware model on a mixture of $\Delta t$ values and then compare its latent predictions to the Lemma 4.1 scaling relation: check whether $d(z,a,\Delta t)$ stays close to $d(z,a,\bar{\Delta t})\cdot \frac{\Delta t}{\tau(\Delta t)}\frac{\tau(\bar{\Delta t})}{\bar{\Delta t}}$ for pairs of step sizes inside the training range; if the difference remains large after training, the interpolation assumption behind sample efficiency is false.

Watch

Extended reading notes

Core claim

The paper claims that a latent world model for model-based reinforcement learning can learn control dynamics across many observation rates at once if every time-dependent component is conditioned on the time step $\Delta t$ and training uses a mixture of step sizes. In the proposed time-aware model the next latent state is computed as $z_{t+\Delta t}=z_t+d(z_t,a_t,\Delta t)\,\tau(\Delta t)$, where $d$ is a learned latent-derivative network and $\tau(\Delta t)=\max(0,\log_{10}\Delta t+5)$ rescales the step; the reward, value, and policy priors also take $\Delta t$ as input. The paper reports that with the same number of training samples and iterations as the fixed-rate baseline, this model matches the baseline's success rate at the default 2.5 ms evaluation step and reaches substantially higher success rates at evaluation steps of 5 to 50 ms across nine manipulation tasks and three PDE-control tasks. The accompanying sample-efficiency argument holds that reducing the model error at a coarse time scale lowers the error bound at every finer scale, because the optimal dynamics function satisfies a step-size interpolation relation.

Load-bearing premise

The sample-efficiency proof assumes the learned dynamics network actually picks up the interpolation relation between step sizes, which is essentially the transfer across time scales that the lemmas are meant to establish; if that interpolation is not learned, the stated guarantee collapses.

Editorial extensions

If this is right

  • A single trained model can be evaluated at observation rates different from its training rate, using one-step predictions rather than repeated small-step rollouts that accumulate error.
  • The same training budget in steps and samples suffices; no additional data collection is needed to gain robustness to coarser observation rates.
  • Fixed-rate baselines trained only at coarse steps can fail to converge on some tasks, while the mixture-trained model succeeds across the whole evaluated range, indicating that rate diversity during training is doing the work.
  • Both Euler and Runge-Kutta instantiations of the time-aware model outperform fixed-rate models, with Runge-Kutta helping more on complex nonlinear dynamics such as the PDE-control tasks.
  • The formulation is architecture-agnostic in the sense that conditioning the dynamics, reward, value, and policy on $\Delta t$ can be added to an existing latent world model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the interpolation transfer holds, the model is effectively learning a continuous-time latent dynamics, so the learned derivative network $d$ should approach the true latent derivative as $\Delta t$ shrinks; directly probing $d$ at $\Delta t\to 0$ would test that interpretation.
  • Performance is only demonstrated inside the sampled $\Delta t$ range, so a stress test at evaluation steps beyond the training maximum would reveal whether the model extrapolates or merely interpolates across time scales.
  • The same conditioning could enable closed-loop adaptive-rate control, where the agent chooses the next observation interval online based on its confidence in the prediction; the paper does not explore that use.
  • A coarse-to-fine training curriculum over $\Delta t$ might improve sample efficiency further, but that schedule is not tested here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Time-Aware World Model (TAWM), an adaptation of TD-MPC2 in which the latent dynamics, reward, value, and policy networks receive the time-step size Δt as an input; the latent dynamics are advanced by Euler or RK4 integration with a log-scaled step τ(Δt); and training episodes draw Δt from a log-uniform distribution over a task-specific range. The authors report that TAWM matches or exceeds a fixed-Δt TD-MPC2 baseline at the default 2.5 ms rate and substantially outperforms it at evaluation rates of 5–50 ms on Meta-World and PDE-control tasks while using the same number of training steps and samples. They also provide a theoretical discussion in Section 4.3 and Appendices H.1–H.2 intended to justify sample efficiency via interpolation across time steps, comparisons against MTS3, and ablations of log-uniform versus uniform Δt sampling.

Significance. TAWM addresses a real limitation of model-based RL agents: deployment at observation rates different from the training rate. If the reported gains are causally due to time-conditioning together with mixture training, the method is simple, architecture-agnostic, and likely useful across robotic and PDE control. The paper deserves credit for evaluating nine Meta-World tasks and three PDE-control tasks, for comparing against fixed-Δt baselines trained at several explicit Δt values, for including an MTS3 comparison, and for releasing code. The main empirical comparisons are internally consistent, but the causal attribution of the gains to time-awareness is underdetermined by the current experiments, and the theoretical sample-efficiency argument rests on an assumption that is close to the conclusion it is meant to prove.

major comments (4)
  1. [Section 4.3 / Appendix H.2] Assumption 3 states that during training the dynamics function d learns the interpolation relationship of Lemma 4.1 because the interpolation factor is shared across samples, and Lemma 4.2 then uses Assumption 3 to bound the small-Δt error by the large-Δt error. This makes the theoretical transfer claim assumption-dependent rather than established: the interpolation property is precisely the cross-timescale generalization the lemmas are supposed to justify. Please either prove the interpolation-learning property from the training objective and initialization, or present the theoretical analysis as a heuristic and remove the proof-based claim. In particular, the proof of Lemma 4.2 in Appendix H.2 absorbs the unproven 2ε term directly from Assumption 3, so it does not independently substantiate sample efficiency.
  2. [Figures 3–5 and Appendix E] All evaluation rates in Figures 3–5 and Appendix E lie inside the training Δt interval [1, 50] ms, so the advantage over the fixed-Δt baselines could be explained by exposure to large-Δt transitions during training rather than by time-aware interpolation. The missing controls are (i) a TAWM trained at fixed Δt = 2.5 ms and evaluated at 5–50 ms, and (ii) a non-time-conditioned TD-MPC2 trained on the same log-uniform Δt mixture as TAWM. Without these two arms, the results do not identify which ingredient—Δt conditioning or data diversity from mixed Δt—produces the reported gains. This is load-bearing because the abstract and introduction attribute the gains specifically to time-awareness.
  3. [Figure 5 / Section 5.1] The sample-efficiency claim that TAWM does not require additional training steps or samples is based on learning curves evaluated at 2.5 ms, where the baseline is trained and TAWM also sees samples, and at 10–50 ms, where TAWM sees training samples but the fixed-Δt baseline does not. These curves therefore do not isolate interpolation or generalization to unseen rates. Please add evaluations at rates outside the training range (for example, Δt = 75 ms or 100 ms for Meta-World) and matched training-data content, or qualify the data-efficiency claim accordingly.
  4. [Appendix A, Eq. (4)] The RK4 update as written appears inconsistent: k1, k2, k3, and k4 all evaluate d at step-size argument Δt, while the intermediate state updates use d at Δt/2 with τ(Δt/2), and the final update multiplies by τ(Δt). As written, this is not the standard RK4 method and is hard to reproduce from the paper. Please correct the equations or clarify the intended variant, and confirm that the released code implements the corrected formulas, since the RK4 results in Figures 3 and 17 depend on this definition.
minor comments (5)
  1. [Section 1, first paragraph of contributions] In the sentence 'In this work, We focus on the following question,' the capitalization of 'We' is inconsistent with the surrounding text.
  2. [Appendix D, first paragraph] The text 'SSM f ast' appears to be a truncated or corrupted token; it should read 'SSM_fast' and the sentence should be completed.
  3. [Figure 6 and Appendix D] The MTS3 comparison uses 4M offline transitions for MTS3 while TAWM uses 1.5M online steps, and MTS3 is combined with TAWM's trained reward and value functions; please clarify whether any sample-efficiency or performance conclusion from this comparison is intended, given that the training budgets and data distributions are not matched.
  4. [Section 5.1, Training Setup] The paper reports that TAWM training takes 40–45 hours on one RTX 4000 GPU but does not report the wall-clock time for the baseline; since the paper claims equal sample and step budget but not equal wall-clock time, please report runtime for both to avoid an implicit efficiency claim.
  5. [Limitations and Future Work] The limitations paragraph acknowledges that Δt_max is determined empirically and that the method has no systematic way to compute the highest frequency of task dynamics; this caveat should be reflected in the abstract's claim that TAWM learns 'both high- and low-frequency task dynamics,' since the appropriate range is task-tuned.

Circularity Check

1 steps flagged · score 4.0 of 10

Theoretical sample-efficiency proof assumes the cross-timescale interpolation it claims to establish; empirical results are independent.

  1. other [Section 4.3, 'Interpolation Learning' assumption and Lemma 4.2; Appendix H.2, Eq. (7)-(8)]
    "Interpolation Learning: During training, the dynamics function d effectively learns the relationship described in Lemma 4.1 as the interpolation factor is shared across all samples. ... Under Assumption 3 in Section 4.3, the following holds for our current dynamics model d: ||d(zt, at, ∆t) − d(zt, at, ∆¯t) · ∆t/τ(∆t) · τ(∆¯t)/∆¯t || < ε."

    Lemma 4.2 is presented as proving that 'error reductions at larger temporal scales contribute to reductions at smaller scales during training,' which is the mechanism behind TAWM's claimed sample efficiency. Its proof, however, begins by assuming Eq. (7), which states that the current learned dynamics model d already satisfies exactly the interpolation relationship of Lemma 4.1. That relationship is the very cross-timescale transfer the theory is supposed to establish. The rest of the proof is a triangle-inequality repackaging of this assumed relation into an error bound, so the theoretical sample-efficiency conclusion is not derived from the training objective or the mixture-of-timescales procedure; it is assumed as the key premise.

full rationale

The paper's main empirical contribution—TAWM conditioned on Δt and trained on a log-uniform mixture of Δt values beating fixed-Δt TD-MPC2 and MTS3 baselines at evaluation rates of 1–50 ms—is supported by head-to-head experiments in Figures 3–6 and Appendices E–F. Those comparisons are self-contained against external baselines and do not reduce to the theory, so the central empirical claim is not circular. The circularity is confined to Section 4.3: Lemma 4.2's proof uses Assumption 3 (Eq. 7) to assert that the learned dynamics model d already exhibits the interpolation relation of Lemma 4.1, which is precisely the transfer behavior that a proof of sample efficiency would need to derive. The paper does label this as an 'assumption,' so the weakness is partly an unproven premise rather than a hidden fit; nevertheless, the 'theoretical insights' claiming to show sample efficiency are conditional on the very effect they purport to explain. I did not find load-bearing self-citation: TD-MPC2 and MTS3 are external prior works, and the authors' earlier citation (Nhu et al., 2023) is not used as evidence for the main claims. The empirically chosen Δt range, acknowledged in the Limitations section, is a correctness/scope limitation rather than a circular step. The observation that all evaluation Δt values lie inside the training range is an experimental-design confound for 'generalization' but not a circularity under the defined patterns. Overall score 4: one substantive circular step in the theoretical derivation, while the central empirical result retains independent content.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The method introduces several hand-set quantities and assumptions. The tau(Delta t) offset and the Delta t bounds are empirically chosen. The theoretical lemmas rely on smoothness of the true dynamics across Delta t and on an interpolation-learning assumption that is close to the conclusion. No new physical entities are introduced.

free parameters (5)
  • tau(Delta t) offset (the '+5' in max(0, log10 Delta t + 5)) = 5
    Hand-chosen to normalize the latent increment scale over the Delta t range; the authors report convergence failures in mw-assembly with linear Delta t integration (Section 4.1.2), so this constant is empirically tuned.
  • Delta t_min (Meta-World) = 0.001 s
    Lower bound of the training time-step range; chosen by hand. The theory assumes the range captures the task dynamics (Assumption 2).
  • Delta t_max (Meta-World) = 0.05 s
    Upper bound of the training time-step range; the authors state there is no systematic way to determine the highest task frequency, so this is set empirically (Section 4.2; Limitations).
  • Delta t range (PDE-control) = 0.01 s to 1.0 s
    Empirically chosen range for PDE tasks, stated in Algorithm 1 comments and Section 5.
  • Delta t sampling distribution = log-uniform (primary), uniform (ablation)
    A tunable hyperparameter; the paper shows uniform sampling can outperform log-uniform on some tasks, so the choice affects performance (Section 5.2).
assumptions (6)
  • domain assumption The environment dynamics decompose into subsystems, each with its own highest frequency, and the Nyquist-Shannon theorem applies to each subsystem in the MBRL setting.
    Motivates the mixture-of-Delta t training; stated in Sections 3.2.1 and 3.2.2 but not established for the control tasks.
  • domain assumption The underlying world-model architecture has enough representation power to capture the system's complex dynamics (Assumption 1).
    Section 4.3, Assumption 1. Standard but unverified for the specific MLP latent dynamics.
  • ad hoc to paper The empirically determined range [Delta t_min = 0.001 s, Delta t_max = 1 s] adequately captures the behavior of most subsystems (Assumption 2).
    Section 4.3, Assumption 2. Ad hoc because Delta t_max is chosen empirically and depends on the task.
  • ad hoc to paper During training, the dynamics function d effectively learns the interpolation relationship of Lemma 4.1 because the interpolation factor is shared across samples (Assumption 3).
    Section 4.3, Assumption 3. This assumption asserts the central mechanism the theory aims to prove, making the theoretical sample-efficiency argument circular.
  • domain assumption The ground-truth dynamics f is essentially independent of Delta t for all Delta t below some Delta t bar, i.e., the dynamics 'can be fully captured with Delta t bar.'
    Definition in Section 4.3 and used in Lemmas 4.1 and 4.2; a strong smoothness condition that fails if the system has high-frequency modes requiring the smallest Delta t.
  • ad hoc to paper The tau(Delta t) = max(0, log10 Delta t + 5) scaling preserves the correctness of Euler/RK4 integration in latent space.
    Section 4.1.2; the logarithmic scaling is introduced for numerical stability but is not derived from the dynamics, so the integrated update is no longer a standard Euler/RK4 discretization of the true latent ODE.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Time-Aware World Model for Adaptive Prediction and Control." pith.science (2026). https://pith.science/paper/JA2P57RO

@misc{pith2026250608441,
  author       = {Pith},
  title        = {Pith review of: Time-Aware World Model for Adaptive Prediction and Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JA2P57RO}},
  note         = {Machine review of arXiv:2506.08441}
}
read the original abstract

In this work, we introduce the Time-Aware World Model (TAWM), a model-based approach that explicitly incorporates temporal dynamics. By conditioning on the time-step size, {\Delta}t, and training over a diverse range of {\Delta}t values -- rather than sampling at a fixed time-step -- TAWM learns both high- and low-frequency task dynamics across diverse control problems. Grounded in the information-theoretic insight that the optimal sampling rate depends on a system's underlying dynamics, this time-aware formulation improves both performance and data efficiency. Empirical evaluations show that TAWM consistently outperforms conventional models across varying observation rates in a variety of control tasks, using the same number of training samples and iterations. Our code can be found online at: github.com/anh-nn01/Time-Aware-World-Model.

Figures

Figures reproduced from arXiv: 2506.08441 by the authors.

Figure 1
Figure 1. Overall framework of our time-aware world model. An encoder h encodes the given observation ot into a latent vector zt, which is then fed into various models with action at and time step size ∆t to estimate values for action planning. interest in model-based RL (MBRL). Unlike model-free approaches, MBRL constructs a model M that captures the underlying task dynamics (Sutton, 1990; Deisenroth & Rasmussen, 2011; Parma… view at source ↗
Figure 2
Figure 2. Visualizations of the Meta-World control tasks. From left to right: Assembly, Basketball, Box Close, Faucet Open. The default time step size of all environments is ∆t = 2.5ms. Lemma 4.2. When the environment’s dynamics can be fully captured with ∆t¯, reducing the modeling error at ∆t¯lowers the upper bound of the error for every ∆t < ∆t¯. Proof: Please see Appendix H.2. Building on this lemma, we conjecture that imp… view at source ↗
Figure 3
Figure 3. Performance comparisons for 3 tasks from Meta-World (Up), and 3 tasks from PDE-Control (Down). The x-axis corresponds to the evaluation ∆t (in ms), and the y-axis corresponds to success rate (Meta-world) and episode reward (PDE-Control). We trained our TAWM model using either RK4 or Euler integration method with log-uniform sampling strategy. The baseline method was trained using only default time step (∆t = 2.5 ms … view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Performance comparisons to baseline methods trained on different ∆t values for Meta-World tasks. The x-axis corresponds to the evaluation ∆t, and the y-axis corresponds to the success rate. The non–time-aware baseline models were trained with different fixed ∆t values …
Figure 5
Figure 5. Figure 5: Learning curve comparison on Meta-World Basketball (Up) and Meta-World Lever-Pull (Down). The x-axis corresponds to the evaluation ∆t (in ms), and the y-axis corresponds to the success rate. We trained our TAWM using either RK4 or Euler integration method with log-unif…
Figure 6
Figure 6. Figure 6: Performance comparisons to MTS3 across different evaluation ∆t’s on Meta-World tasks. The x-axis corresponds to the evaluation ∆t, and the y-axis corresponds to the success rate. The MTS3 models were trained under different slow dynamics set￾tings, indicated by the H v…
Figure 7
Figure 7. Figure 7: shows that while TAWM trained with the uniform sampling strategy generally performs better in most envi￾ronments and achieves higher performance at low sampling rates (∆t ≥ 30 ms), it exhibits lower success rates at smaller inference ∆t in some environments, such as Me…
Figure 8
Figure 8. Figure 8: shows that while our time-aware models trained with the uniform sampling strategy generally perform better across most environments – and significantly outperform at low sampling rates (inference ∆t ≥ 30 ms) – they exhibit lower success rates at smaller inference ∆t in…
Figure 9
Figure 9. Figure 9: Visualizations of the Meta-World control tasks. From top to bottom: (1) Assembly, (2) Basketball, (3) Box Close, (4) Faucet Open, (5) Hammer. From left to right: sequences of renderings from task initialization to completion. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Visualizations of the Meta-World control tasks. From top to bottom: (6) Handle Pull, (7) Lever Pull, (8) Pick Out Of Hole, (9) Sweep Into. From left to right: sequences of renderings from task initialization to completion. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Performance comparisons for 6 additional tasks from Meta-World tasks. The x-axis corresponds to the evaluation ∆t (in ms), and the y-axis corresponds to success rate. We trained our TAWM model using either RK4 or Euler integration method with log-uniform sampling stra…
Figure 12
Figure 12. Figure 12: Additional Meta-World tasks: Success Rate Curve under different evaluation time step sizes. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Additional Meta-World tasks: Success Rate Curve under different evaluation time step sizes. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Visualizations of systems dynamics of uncontrolled Burgers PDE. The spatial domain has length L = 1 with the diffusivity (viscosity) parameter ν = 10−3 . The initial state is u(x, t = 0) = sech(10x − 5). Burgers’ equation is a simplified PDE that captures the essentia…
Figure 15
Figure 15. Figure 15: Visualizations of systems dynamics of uncontrolled Allen-Cahn PDE. The spatial domain has length L = 2 with the diffusivity (viscosity) parameter ν = 10−4 and potential constant V = 5.0. The initial state is u(x, t = 0) = (x − 1)2 · cos(π(x − 1)). G.3. Allen-Cahn Equa…
Figure 16
Figure 16. Figure 16: Visualizations of systems dynamics of uncontrolled Wave PDE. The spatial domain has length L = 1 with c = 0.1. The initial state is u(x, t = 0) = sech(10x − 5) and ψ(x, t = 0) = 0. The wave equation is a second-order linear PDE describing the spatial propagation of wa…
Figure 17
Figure 17. Figure 17: PDE control tasks: Average Negative LQ Error learning curves under different evaluation time step sizes. The default time step size are ∆t = 50 ms/10 ms/100 ms for PDE-Burgers, PDE-Allen-Cahn, PDE-Wave, respectively. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_17.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OpenWorldLib: A Unified Codebase and Definition of Advanced World Models

    cs.CV 2026-04 unverdicted novelty 4.0 of 10

    OpenWorldLib offers a standardized codebase and definition for world models that combine perception, interaction, and memory to understand and predict the world.

Reference graph

Works this paper leans on

47 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    G., Sutton, R

    Barto, A. G., Sutton, R. S., and Anderson, C. W. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 0 (5): 0 834--846, 1983

    work page 1983

  3. [3]

    Butcher, J. C. The numerical analysis of ordinary differential equations: Runge-Kutta and general linear methods. Wiley-Interscience, 1987

    work page 1987

  4. [4]

    and Rasmussen, C

    Deisenroth, M. and Rasmussen, C. E. Pilco: A model-based and data-efficient approach to policy search. In Proceedings of the 28th International Conference on Machine Learning (ICML), pp.\ 465--472, 2011

    work page 2011

  5. [5]

    P., Neumann, G., Peters, J., et al

    Deisenroth, M. P., Neumann, G., Peters, J., et al. A survey on policy search for robotics. Foundations and Trends in Robotics , 2 0 (1--2): 0 1--142, 2013

    work page 2013

  6. [6]

    World models for autonomous driving: An initial survey

    Guan, Y., Liao, H., Li, Z., Hu, J., Yuan, R., Li, Y., Zhang, G., and Xu, C. World models for autonomous driving: An initial survey. IEEE Transactions on Intelligent Vehicles, 2024

    work page 2024

  7. [7]

    and Schmidhuber, J

    Ha, D. and Schmidhuber, J. World models. arXiv preprint arXiv:1803.10122, 2018

    arXiv 2018

  8. [8]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning (ICML), pp.\ 1861--1870. PMLR, 2018

    work page 2018

Show all 47 references
  1. [9]

    Dream to control: Learning behaviors by latent imagination

    Hafner, D., Lillicrap, T., Ba, J., and Norouzi, M. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603, 2019

    1912 arXiv

  2. [10]

    Mastering atari with discrete world models

    Hafner, D., Lillicrap, T., Norouzi, M., and Ba, J. Mastering atari with discrete world models. arXiv preprint arXiv:2010.02193, 2020

    2010 arXiv

  3. [11]

    Mastering diverse domains through world models

    Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023

    2023 arXiv

  4. [12]

    TD - MPC 2: Scalable, robust world models for continuous control

    Hansen, N., Su, H., and Wang, X. TD - MPC 2: Scalable, robust world models for continuous control. In The Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  5. [13]

    A., Su, H., and Wang, X

    Hansen, N. A., Su, H., and Wang, X. Temporal difference learning for model predictive control. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.), Proceedings of the 39th International Conference on Machine Learning (ICML), volume 162, pp.\...

    2022

  6. [14]

    When to trust your model: Model-based policy optimization

    Janner, M., Fu, J., Zhang, M., and Levine, S. When to trust your model: Model-based policy optimization. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019

    2019

  7. [15]

    Jerri, A. J. The shannon sampling theorem—its various extensions and applications: A tutorial review. Proceedings of the IEEE, 65 0 (11): 0 1565--1596, 1977

    1977

  8. [16]

    A., Solowjow, E., and Levine, S

    Johannink, T., Bahl, S., Nair, A., Luo, J., Kumar, A., Loskyll, M., Ojea, J. A., Solowjow, E., and Levine, S. Residual reinforcement learning for robot control. In 2019 International Conference on Robotics and Automation (ICRA), pp.\ 6023--6029. IEEE, 2019

    2019

  9. [17]

    H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., et al

    Kaiser, L., Babaeizadeh, M., Milos, P., Osinski, B., Campbell, R. H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., et al. Model-based reinforcement learning for atari. arXiv preprint arXiv:1903.00374, 2019

    1903 arXiv

  10. [18]

    R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A

    Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A. A., Yogamani, S., and P \'e rez, P. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23 0 (6): 0 4909--4926, 2021

    2021

  11. [19]

    Y., Lee, H., Yang, Y., Baldridge, J., and Anderson, P

    Koh, J. Y., Lee, H., Yang, Y., Baldridge, J., and Anderson, P. Pathdreamer: A world model for indoor navigation, 2021. URL https://arxiv.org/abs/2105.08756

    2021 arXiv

  12. [20]

    Investigating compounding prediction errors in learned dynamics models, 2022

    Lambert, N., Pister, K., and Calandra, R. Investigating compounding prediction errors in learned dynamics models, 2022. URL https://arxiv.org/abs/2203.09637

    2022 arXiv

  13. [21]

    Crafting papers on machine learning

    Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp.\ 1207--1216, Stanford, CA, 2000. Morgan Kaufmann

    2000

  14. [22]

    Learning dynamics models for model predictive agents, 2021

    Lutter, M., Hasenclever, L., Byravan, A., Dulac-Arnold, G., Trochim, P., Heess, N., Merel, J., and Tassa, Y. Learning dynamics models for model predictive agents, 2021. URL https://arxiv.org/abs/2109.14311

    2021 arXiv

  15. [23]

    N., Le, N.-A., Li, S., and Truong, T

    Nhu, A. N., Le, N.-A., Li, S., and Truong, T. D. Physics-guided reinforcement learning system for realistic vehicle active suspension control. In 2023 International Conference on Machine Learning and Applications (ICMLA), pp.\ 422--429. IEEE, 2023

    2023

  16. [24]

    E., Peters, J., and Doya, K

    Parmas, P., Rasmussen, C. E., Peters, J., and Doya, K. Pipps: Flexible model-based policy search robust to the curse of chaos. In International Conference on Machine Learning (ICML), pp.\ 4065--4074. PMLR, 2018

    2018

  17. [25]

    Prasad, N., Cheng, L.-F., Chivers, C., Draugelis, M., and Engelhardt, B. E. A reinforcement learning approach to weaning of mechanical ventilation in intensive care units. arXiv preprint arXiv:1704.06300, 2017

    2017 arXiv

  18. [26]

    Trust region policy optimization

    Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. Trust region policy optimization. In International Conference on Machine Learning (ICML), pp.\ 1889--1897. PMLR, 2015

    2015

  19. [27]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  20. [28]

    Multi time scale world models

    Shaj Kumar, V., Gholam Zadeh, S., Demir, O., Douat, L., and Neumann, G. Multi time scale world models. Advances in Neural Information Processing Systems (NeurIPS), 36: 0 26764--26775, 2023

    2023

  21. [29]

    Communication in the presence of noise

    Shannon, C. Communication in the presence of noise. Proceedings of the IRE , 37 0 (1): 0 10--21, jan 1949. doi:10.1109/jrproc.1949.232969. URL https://doi.org/10.1109/jrproc.1949.232969

    1949

  22. [30]

    J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al

    Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. Mastering the game of go with deep neural networks and tree search. Nature, 529 0 (7587): 0 484--489, 2016

    2016

  23. [31]

    Mastering the game of go without human knowledge

    Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. Mastering the game of go without human knowledge. Nature, 550 0 (7676): 0 354--359, 2017

    2017

  24. [32]

    A general reinforcement learning algorithm that masters chess, shogi, and go through self-play

    Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., and Hassabis, D. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science...

    2018 doi

  25. [33]

    Gradient informed proximal policy optimization

    Son, S., Zheng, L., Sullivan, R., Qiao, Y.-L., and Lin, M. Gradient informed proximal policy optimization. Advances in Neural Information Processing Systems (NeurIPS), 36, 2024

    2024

  26. [34]

    J., Simchowitz, M., Zhang, K., and Tedrake, R

    Suh, H. J., Simchowitz, M., Zhang, K., and Tedrake, R. Do differentiable simulators give better policy gradients? In International Conference on Machine Learning (ICML), pp.\ 20668--20696. PMLR, 2022

    2022

  27. [35]

    Sutton, R. S. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Machine learning proceedings 1990, pp.\ 216--224. Elsevier, 1990

    1990

  28. [36]

    S., McAllester, D., Singh, S., and Mansour, Y

    Sutton, R. S., McAllester, D., Singh, S., and Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. In Solla, S., Leen, T., and M\" u ller, K. (eds.), Advances in Neural Information Processing Systems, volume 12. MIT Press, 1999

    1999

  29. [37]

    Thodoroff, P., Li, W., and Lawrence, N. D. Benchmarking real-time reinforcement learning. In Albanie, S., Henriques, J. F., Bertinetto, L., Hernández-Garcı́a, A., Doughty, H., and Varol, G. (eds.), NeurIPS 2021 Workshop on Pre-registration in Machine Learning, volume 181, pp.\...

    2021

  30. [38]

    M., Mathieu, M., Dudzik, A., Chung, J., Choi, D

    Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575 0 (7782): 0 350--354, 2019

    2019

  31. [39]

    Benchmarking model-based reinforcement learning, 2019

    Wang, T., Bao, X., Clavera, I., Hoang, J., Wen, Y., Langlois, E., Zhang, S., Zhang, G., Abbeel, P., and Ba, J. Benchmarking model-based reinforcement learning, 2019. URL https://arxiv.org/abs/1907.02057

    2019 arXiv

  32. [40]

    Principles of multiscale modeling

    Weinan, E. Principles of multiscale modeling. Cambridge University Press, 2011

    2011

  33. [41]

    Williams, R. J. and Peng, J. Reinforcement learning algorithms as function optimizers. In Proceedings of the International Joint Conference on Neural Networks, Washington DC, volume 2, pp.\ 89--95, 1989

    1989

  34. [42]

    Daydreamer: World models for physical robot learning

    Wu, P., Escontrela, A., Hafner, D., Abbeel, P., and Goldberg, K. Daydreamer: World models for physical robot learning. In Conference on Robot Learning (CoRL), pp.\ 2226--2240. PMLR, 2023

    2023

  35. [43]

    Accelerated policy learning with parallel differentiable simulation

    Xu, J., Makoviychuk, V., Narang, Y., Ramos, F., Matusik, W., Garg, A., and Macklin, M. Accelerated policy learning with parallel differentiable simulation. In International Conference on Learning Representations (ICLR), 2022

    2022

  36. [44]

    Reinforcement learning for optimal control of low exergy buildings

    Yang, L., Nagy, Z., Goffin, P., and Schlueter, A. Reinforcement learning for optimal control of low exergy buildings. Applied Energy, 156: 0 577--586, 2015

    2015

  37. [45]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning

    Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Kaelbling, L. P., Kragic, D., and Sugiura, K. (eds.), Proceedings of the Conference on Robot Learning (CoRL)...

    2020

  38. [46]

    A sampling theorem for exact identification of continuous-time nonlinear dynamical systems

    Zeng, Z., Yue, Z., Mauroy, A., Gonçalves, J., and Yuan, Y. A sampling theorem for exact identification of continuous-time nonlinear dynamical systems. IEEE Transactions on Automatic Control, pp.\ 1--16, 2024. doi:10.1109/TAC.2024.3409639

    2024

  39. [47]

    Controlgym: Large-scale control environments for benchmarking reinforcement learning algorithms

    Zhang, X., Mao, W., Mowlavi, S., Benosman, M., and Ba s ar, T. Controlgym: Large-scale control environments for benchmarking reinforcement learning algorithms. In 6th Annual Learning for Dynamics & Control Conference, pp.\ 181--196. PMLR, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.