Pith. sign in

REVIEW 4 major objections 6 minor 45 references

Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that long-horizon world-model accuracy is driven by the training objective—end-to-end endpoint prediction—not by the choice of backbone architecture.

desk verdict Long-horizon endpoint supervision is a real and testable idea, and the ADM retraining is the right experiment, but the evidence is thinner than the abstract claims: one task, no error bars, and missing short-horizon baselines. read the letter →

arxiv 2608.07420 v1 pith:AP6DZWU6 submitted 2026-08-07 cs.LG

Xinyi Li , Zaishuo Xia , Chenjie Hao , Yubei Chen This is my paper · ORCID
classification cs.LG
keywords worldmodelslong-horizonpredictiondirectendpointnon-recursivedynamicscompoundingerrortrainingobjectivemodel-basedreinforcementlearningend-to-endsupervision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

World models are usually trained to predict one step ahead and then rolled out recursively, so small errors compound over long horizons. This paper argues that long-horizon accuracy should be optimized directly: attach the loss to the endpoint observation reached after an entire action sequence, and train end-to-end. To make that practical, it introduces DPWM, a non-recursive model that compresses the action sequence into a single embedding and predicts the endpoint in one forward pass, so gradient depth stays fixed no matter how long the horizon. The central claim is that the training objective, rather than any particular backbone, is the main driver of long-horizon accuracy—supported by retraining a recurrent baseline under the same endpoint objective and seeing its gap to DPWM mostly close.

What carries the argument

The load-bearing object is the $K$-step endpoint prediction objective $\mathcal{L}(\theta) = \mathbb{E}\left[ \| D(g(E(o_t), \phi(a_{t:t+K-1}))) - o_{t+K}\|^2 \right]$ with $K$ sampled from a horizon distribution. The direct prediction model $g_\theta$ is non-recursive: an action-sequence encoder $\phi$ (a bidirectional windowed Transformer with rotary position embeddings, mean-pooled) turns the whole action list into one embedding, and a FiLM-conditioned residual MLP maps the initial observation latent to the endpoint latent in a single pass. Two mechanisms carry the argument: the error bound comparison (recursive rollout amplifies by $\epsilon \sum_{i=0}^{K-1} L^i$ while direct prediction pays only $\delta_K$) and the gradient-alignment identity in Eq. (5), where the endpoint's parameter sensitivity weights each transition by the Jacobian product $J_{k\to K}$ and the endpoint loss reproduces that weighting automatically. The architecture's role is enabler, not source of accuracy: it keeps gradient depth fixed so long-horizon end-to-end training is tractable.

What would settle it

Reproduce Table 2 by training ADM at $K_{\max}=100$ using exactly the published $K=3$ hyperparameters and inference procedure; if endpoint MSE at horizons 100 and 200 does not fall clearly below the $K=3$ numbers, the central claim that the training objective drives long-horizon accuracy is not supported. A complementary check is to train a one-step autoregressive world model with the same endpoint objective via backpropagation through time and see whether its long-horizon endpoint error drops to DPWM levels.

Watch

Extended reading notes

Core claim

The paper's central claim is that long-horizon prediction accuracy in world models is largely determined by the training objective and prediction paradigm, not by the specific architecture. Formally, a $K$-step transition map $f^{(K)}$ can be approximated either by composing a one-step model $\hat f$ and rolling it out, which gives an endpoint error bounded by $\epsilon \sum_{i=0}^{K-1} L^i$ and amplifies perturbations whenever the dynamics are expansive ($L>1$), or by learning a direct approximator $g_\theta$ whose endpoint error is just $\delta_K = \sup \|g_\theta - f^{(K)}\|$ with no recursive amplification. Training $g_\theta$ with an endpoint loss over sampled horizons attaches gradients to the endpoint, so each transition is weighted by its downstream Jacobian influence exactly as in the sensitivity of the endpoint; the one-step objective weights all transitions uniformly and is misaligned with that sensitivity. DPWM instantiates this paradigm non-recursively—a Transformer action-sequence encoder plus a FiLM-conditioned dynamics module—so the gradient path from the loss to any parameter has fixed depth regardless of $K$. Consistent with the objective-first claim, retraining the recurrent baseline ADM with the same long-horizon endpoint objective substantially closes its gap to DPWM, while its published short-horizon training does poorly at long horizons.

Load-bearing premise

The load-bearing premise is that the ADM $K=100$ versus $K=3$ comparison isolates the training horizon—same encoder, same inference procedure, stable optimization at $K=100$—so that if long-horizon training required different hyperparameters or changed inference, the conclusion that objective rather than backbone drives accuracy would be confounded.

Editorial extensions

If this is right

  • If the objective is the main driver, world models should be trained and evaluated at the temporal horizon where they will actually be used; the paper shows $K_{\max}=50$ degrades beyond its training horizon while $K_{\max}=100$ stays accurate at $K=200$.
  • Recurrent or autoregressive models can also benefit: training them with an end-to-end endpoint objective (backpropagating through the unrolled rollout) should inherit the Jacobian-weighted gradient and improve long-horizon accuracy, as the ADM retraining experiment shows.
  • Direct endpoint prediction removes the recursive feedback path at inference, so compounding error from self-conditioning on predicted states is avoided even before considering the training objective.
  • A model trained only on endpoint supervision can still supply intermediate frames by querying successive action prefixes in separate forward passes, making it usable for trajectory-level planning without autoregressive rollout.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension this paper leaves implicit is that the objective-first view predicts any sufficiently expressive backbone trained end-to-end at the same horizon will match DPWM; a direct test would fix the endpoint loss and vary only the sequence encoder.
  • A testable consequence for model-based RL is that evaluation should use endpoint error at the planning horizon rather than average rollout error, since the paper's own metric choice signals this shift.
  • The deterministic formulation does not cover stochastic dynamics; extending the endpoint objective to conditional distributions or conditional means is a natural next step this paper names as future work.
  • The non-recursive design also suggests a planning advantage: many candidate action sequences can be evaluated in parallel single forward passes, which the paper does not exploit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript argues that world models are conventionally trained with local one-step or few-step objectives and then evaluated by recursive rollout, which mismatches the long-horizon use case. It proposes the Direct Prediction World Model (DPWM), a non-recursive architecture that encodes the action sequence into a single embedding and predicts the endpoint observation in one forward pass. The paper derives a Lipschitz-based error bound motivating direct prediction (Appendix A), describes a practical architecture with standard components, and reports endpoint MSE on four DeepMind Control Suite tasks and on Pong. A controlled experiment retrains the ADM baseline under the same endpoint objective (Table 2) and is used to support the central claim that the training objective, rather than the backbone, drives long-horizon prediction accuracy.

Significance. If the central claim survives closer empirical scrutiny, the paper makes a useful contribution: it identifies a train/evaluation mismatch in world modeling, proposes a clean paradigm, and gives a practical architectural pattern that is not tied to a specific backbone. The Lipschitz derivation in Appendix A is clean and is correctly framed as a motivation rather than a guarantee. The experimental design of retraining ADM under Kmax=100 is the right way to isolate objective from architecture, and the qualitative Pong rollouts are suggestive. The main weakness is that the empirical support for the abstract-level claim is currently narrow: the controlled ablation is single-task, main tables lack a pure one-step autoregressive baseline, and no uncertainty quantification is reported anywhere.

major comments (4)
  1. [§3.2, Table 2] The central claim of the paper is supported by a single-task ablation without uncertainty quantification. Table 2 compares DPWM K=100, ADM K=100, and ADM K=3 on humanoid_walk only; at K=1 ADM K=3 is actually better than ADM K=100 (0.1385 vs. 0.3448), and no confidence intervals or repeated-seed results are reported, so the K>=16 advantage cannot be judged statistically significant. The statement that ADM K=3 and ADM K=100 'share the same encoder and inference procedure, differing only in the training horizon' is asserted but not verified: no learning-rate schedule, gradient-clipping configuration, early-stopping criterion, or training curves are reported for the K=100 retraining, and long-horizon unrolled training may require different hyperparameters. Please add multiple seeds with error bars, at least two additional tasks, and full tuning details for the retrained baseline.
  2. [§3.1, Table 1] The main results table lacks a pure one-step autoregressive world-model baseline. Section 3.1 says 'Autoregressive baselines are trained with local one-step prediction losses,' but Table 1 contains only DPWM K=100, ADM K=3, and MoSim; ADM K=3 is a short-horizon any-step model, not a one-step recurrent model, and MoSim is a physics-informed ODE model. Consequently Table 1 does not test the objective-versus-backbone claim at all; only Table 2 does, and only on one task. Please add a recurrent one-step baseline (e.g., ADM K=1 or a standard RSSM) evaluated by autoregressive rollout, or explicitly reframe the headline comparison as an accuracy comparison rather than an objective comparison.
  3. [§3.3 and Appendix D] The pixel-based Pong experiment does not test the training-objective claim. Figure 4 is qualitative, and Table 4 compares DPWM K=100 with ADM K=100, both of which are trained under the same long-horizon endpoint objective; this comparison controls for backbone but cannot reveal whether endpoint supervision is better than short-horizon training. The paper therefore has no short-horizon baseline on the second benchmark, so the abstract-level generalization is untested on pixels. Please add a short-horizon (K=1 or K=3) recurrent baseline on Pong with rollout evaluation, and report seeds and error bars for Table 4.
  4. [§3.2, Figure 3] The Kmax selection results are presented as single curves without uncertainty, yet they are used to draw conclusions such as 'Kmax=50 performs well only within its training horizon' and 'Kmax=400 yields consistently lower error' under random-policy evaluation. These claims would be substantially strengthened by reporting at least three seeds with confidence bands, and by specifying the horizon sampling distribution (uniform vs. log-uniform) and the random-segment intermix ratio, which are design choices that directly affect the curves.
minor comments (6)
  1. [§3.1] The sentence introducing 'one-step prediction models' is dangling, because no such model appears in Table 1; please either add the baseline or revise the text so the reader can identify which rows correspond to one-step training.
  2. [§2.2, Eq. (5)] The gradient expansion in Eq. (5) is informal because the intermediate states s_j are themselves functions of theta; it would be helpful to state explicitly that this is a schematic unrolled-gradient expression rather than a complete derivation.
  3. [Appendix A] The sup-norm error bound in Eq. (9) is a useful motivation, but the connection to the experiments would be stronger if the analysis were also phrased in expectation over the data distribution, since the reported results are MSE averages over trajectories.
  4. [§3.2, Table 3] The horizon distribution (uniform vs. log-uniform) and the random-segment intermix ratio are mentioned as design choices but are never given numerical values; specifying them would improve reproducibility.
  5. [Reproducibility] The paper does not state whether code, trained models, or random seeds will be released; please add a reproducibility statement and a seed list.
  6. [§3.3 and Appendix B] The repeated independent queries used to produce per-step frames are described clearly, but the text should emphasize more visibly that independently sampled frames are not constrained to be temporally consistent, a point the paper itself acknowledges only in the limitations section.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation found; the central objective-vs-backbone claim is tested by a controlled retraining ablation, though that ablation is reported without error bars or hyperparameter details.

full rationale

The paper's central claim is that long-horizon accuracy is driven by the training objective rather than the backbone. This is tested empirically by retraining ADM under the same long-horizon endpoint objective (ADM K=100) and comparing it to ADM K=3, with the paper explicitly stating that ADM K=3 and ADM K=100 'share the same encoder and inference procedure, differing only in the training horizon.' That is an architecture-controlled comparison, not a reduction of the conclusion to the paper's own architecture. The theoretical comparison in Eq. (10) is explicitly conditional ('whenever δK < ...') and the paper states that 'the substantive question is therefore empirical,' so it is a definitional bound comparison rather than a circular derivation. The few self-citations (SmallWorlds [26], Cloning Deterministic Worlds [40]) appear only as contextual references in the introduction and are not load-bearing; MoSim [21] is an external baseline rather than a self-supporting premise. The K=100 retraining details are not fully reported (no seeds, error bars, or hyperparameter search), which is a rigor concern but not circularity. Accordingly, the score reflects only minor non-load-bearing self-citations.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim is empirically grounded in controlled experiments and standard error-bound analysis. The listed axioms are background assumptions about the environments, the modeling capacity of the non-recursive encoder, and the evaluation metric. No fitted parameters appear in the theoretical derivation; the free parameters are experimental design choices that shape the empirical demonstration and should be reported for reproducibility.

free parameters (3)
  • Maximum training horizon Kmax = 100
    Chosen from {50, 100, 400} based on 'balanced performance across horizons' (Fig. 3); all main experiments use this value, and the long-horizon benefit is demonstrated under this choice.
  • Horizon sampling distribution
    Sec. 2.2 states the horizon K is drawn from 'a predefined horizon distribution, instantiated as either uniform or log-uniform', but the main experiments never specify which; this choice affects the training signal at each horizon.
  • Intermix random-segment ratio
    Sec. 3.2 describes an 'intermix protocol that injects short random segments into expert rollouts' but gives no ratio or segment length; it shapes the training data distribution for policy-evaluation results.
assumptions (5)
  • domain assumption The transition dynamics f are L-Lipschitz in the state argument (Eq. 6 in App. A).
    Used to derive the recursive rollout error bound e_K <= epsilon * sum L^i; real control tasks may lack a global Lipschitz constant, especially with contacts.
  • domain assumption The one-step model error epsilon is bounded uniformly over all states and actions.
    App. A assumes sup ||f_hat - f|| <= epsilon; learned models typically have non-uniform error, and the bound is a worst-case characterization.
  • ad hoc to paper A fixed-size action embedding (mean-pooled RoPE Transformer) can preserve enough information from an arbitrary-length action sequence to predict the endpoint.
    Sec. 2.3 relies on this for DPWM; no capacity or information-theoretic guarantee is provided, and the local attention window (size 20) may limit long-range action dependencies.
  • domain assumption Endpoint MSE at a single horizon is the appropriate metric for long-horizon prediction quality.
    The paper trains and evaluates on endpoint MSE; the Limitations section concedes that trajectory-level consistency and downstream control are not evaluated, so the claim may not transfer to those settings.
  • domain assumption The intermix-policy dataset provides a training distribution diverse enough to support generalization to held-out trajectories and out-of-distribution action sequences.
    The policy-evaluation results depend on this; no analysis of dataset diversity or coverage is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction." pith.science (2026). https://pith.science/paper/AP6DZWU6

@misc{pith2026260807420,
  author       = {Pith},
  title        = {Pith review of: Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AP6DZWU6}},
  note         = {Machine review of arXiv:2608.07420}
}
read the original abstract

World models are expected to support imagination over extended temporal horizons, yet most are still trained through local few-step prediction objectives and deployed by recursively rolling out their own predictions. This creates a fundamental mismatch: few-step losses optimize local transition fidelity, while long-horizon prediction depends on how errors and gradients propagate through the entire trajectory. As a result, transitions with different downstream influence on the endpoint are treated uniformly during training, and small local errors are amplified through recursive inference. We argue that long-horizon accuracy is better achieved by optimizing directly, through an end-to-end endpoint prediction objective. To instantiate this paradigm, we introduce the Direct Prediction World Model (DPWM), a non-recursive architecture that compresses an action sequence of arbitrary length into a single embedding and predicts the endpoint observation in a single forward pass. This design avoids recurrent rollout in both prediction and gradient propagation, making long-horizon end-to-end training practical at horizons where unrolled autoregressive training becomes unstable. Empirically, DPWM substantially improves long-horizon endpoint prediction over recursive world-model baselines on continuous-control and pixel-based benchmarks, with larger gains as the prediction horizon increases. We further show that recurrent baselines benefit similarly when retrained with the same long-horizon endpoint objective, supporting our central claim that the training objective, rather than the particular backbone choice, is the main driver of long-horizon prediction accuracy. Our results suggest that world models can benefit from being trained and evaluated at the temporal scales where they are ultimately used, shifting the focus from local transition modeling toward long-horizon predictive accuracy.

Figures

Figures reproduced from arXiv: 2608.07420 by the authors.

Figure 1
Figure 1. DPWM architecture. Action Sequence Encoder compresses the variable-length action sequence into a single embedding, which conditions a Dynamic Model that directly predicts the endpoint observation oˆt+K from the initial observation ot. Modules are trained end-to-end under a long-horizon reconstruction objective on the endpoint, without generating intermediate observations. away from the training distribution and ampl… view at source ↗
Figure 2
Figure 2. Long-horizon prediction visualizations rendered from states predicted by DPWM [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Endpoint prediction MSE computed in normalized state space across evaluation horizons for models trained with Kmax ∈ {50, 100, 400} on intermix-policy data. Results are shown for humanoid_walk evaluated on intermix-policy (left), cheetah_run on intermix-policy (middle), and cheetah_run on random-policy (right). DPWM is trained with our long-horizon end-to-end endpoint objective using Kmax = 100. ADM is trained using… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: DPWM trajectory generation on Pong. Top: ground-truth observations. Bottom: per-step [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Full DPWM 70-step trajectory generation on Pong. Top: ground-truth observations. Bottom: [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 15 canonical work pages

  1. [1]

    DIAMOND: Diffusion for world modeling: Visual details matter in atari

    Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and François Fleuret. DIAMOND: Diffusion for world modeling: Visual details matter in atari. In Advances in Neural Information Processing Systems, 2024

    work page 2024

  2. [2]

    Kavosh Asadi, Dipendra Misra, and Michael L. Littman. Lipschitz continuity in model-based reinforcement learning, 2018. URLhttps://arxiv.org/abs/1804.07193

    arXiv 2018

  3. [3]

    Kavosh Asadi, Dipendra Misra, Seungchan Kim, and Michel L. Littman. Combating the compounding-error problem with a multi-step model, 2019. URL https://arxiv.org/abs/ 1905.13320

    arXiv 2019

  4. [4]

    M. G. Bellemare, Y . Naddaf, J. Veness, and M. Bowling. The arcade learning environment: An evaluation platform for general agents.Journal of Artificial Intelligence Research, 47:253–279,

    work page

  5. [5]

    Multi-timestep models for model-based reinforcement learning, 2024

    Abdelhakim Benechehab, Giuseppe Paolo, Albert Thomas, Maurizio Filippone, and Balázs Kégl. Multi-timestep models for model-based reinforcement learning, 2024. URL https: //openreview.net/forum?id=Rh4DmXaf8R

    work page 2024

  6. [6]

    Scheduled sampling for sequence prediction with recurrent neural networks, 2015

    Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks, 2015. URL https://arxiv.org/abs/ 1506.03099

    arXiv 2015

  7. [7]

    Bengio, P

    Y . Bengio, P. Simard, and P. Frasconi. Learning long-term dependencies with gradient descent is difficult.IEEE Transactions on Neural Networks, 5(2):157–166, 1994. doi: 10.1109/72.279181

    doi:10.1109/72.279181 1994

  8. [8]

    Genie: Generative interactive environments, 2024

    Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Si...

    arXiv 2024

Show all 45 references
  1. [9]

    Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors,Advances in Neural Information Processing Systems, volume 31. Curran...

    2018

  2. [10]

    Diversity is all you need: Learning skills without a reward function, 2018

    Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function, 2018. URL https://arxiv.org/abs/1802. 06070

    2018

  3. [11]

    Goodfellow, and Sergey Levine

    Chelsea Finn, Ian J. Goodfellow, and Sergey Levine. Unsupervised learning for physical interaction through video prediction.ArXiv, abs/1605.07157, 2016. URL https://api. semanticscholar.org/CorpusID:2659157

    2016 arXiv

  4. [12]

    Professor forcing: a new algorithm for training recurrent networks

    Anirudh Goyal, Alex Lamb, Ying Zhang, Saizheng Zhang, Aaron Courville, and Yoshua Bengio. Professor forcing: a new algorithm for training recurrent networks. InProceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, page 4608–4616, R...

    2016

  5. [13]

    Christian Gumbsch, Noor Sajid, Georg Martius, and Martin V . Butz. Learning hierarchical world models with adaptive temporal abstractions from discrete latent dynamics. InThe Twelfth International Conference on Learning Representations, 2024. URL https://openreview. net/forum?...

    2024

  6. [14]

    World models.arXiv preprint arXiv:1803.10122, 2018

    David Ha and Jürgen Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2018

    2018 arXiv

  7. [15]

    Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019. 10

    1912 arXiv

  8. [16]

    Learning latent dynamics for planning from pixels

    Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. InProceedings of the 36th International Conference on Machine Learning, volume 97 ofProceedings of Machine Learning Res...

    2019

  9. [17]

    Mastering atari with discrete world models.arXiv preprint arXiv:2010.02193, 2020

    Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models.arXiv preprint arXiv:2010.02193, 2020

    2010 arXiv

  10. [18]

    Mastering diverse control tasks through world models.Nature, pages 1–7, 2025

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models.Nature, pages 1–7, 2025

    2025

  11. [19]

    Temporal difference learning for model predictive control, 2022

    Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control, 2022. URLhttps://arxiv.org/abs/2203.04955

    2022 arXiv

  12. [20]

    Td-mpc2: Scalable, robust world models for continuous control, 2024

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control, 2024. URLhttps://arxiv.org/abs/2310.16828

    2024 arXiv

  13. [21]

    Neural motion simulator pushing the limit of world models in reinforcement learning

    Chenjie Hao, Weyl Lu, Yifan Xu, and Yubei Chen. Neural motion simulator pushing the limit of world models in reinforcement learning. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 27608–27617, June 2025

    2025

  14. [22]

    Model-based planning with discrete and continuous actions.arXiv preprint arXiv:1705.07177, 2017

    Mikael Henaff, William F Whitney, and Yann LeCun. Model-based planning with discrete and continuous actions.arXiv preprint arXiv:1705.07177, 2017

    2017 arXiv

  15. [23]

    Efros, and Sergey Levine

    Dinesh Jayaraman, Frederik Ebert, Alexei A. Efros, and Sergey Levine. Time-agnostic pre- diction: Predicting predictable video frames, 2018. URL https://arxiv.org/abs/1808. 07784

    2018

  16. [24]

    How far is video generation from world model: A physical law perspective, 2025

    Bingyi Kang, Yang Yue, Rui Lu, Zhijie Lin, Yang Zhao, Kaixin Wang, Gao Huang, and Jiashi Feng. How far is video generation from world model: A physical law perspective, 2025. URL https://arxiv.org/abs/2411.02385

    2025 arXiv

  17. [25]

    A path towards autonomous machine intelligence version 0.9

    Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62(1):1–62, 2022

    2022

  18. [26]

    Smallworlds: Assessing dynamics understanding of world models in isolated environments, 2025

    Xinyi Li, Zaishuo Xia, Weyl Lu, Chenjie Hao, and Yubei Chen. Smallworlds: Assessing dynamics understanding of world models in isolated environments, 2025. URL https:// arxiv.org/abs/2511.23465

    2025

  19. [27]

    Any-step dynamics model improves future predictions for online and offline reinforcement learning

    Haoxin Lin, Yu-Yan Xu, Yihao Sun, Zhilong Zhang, Yi-Chen Li, Chengxing Jia, Junyin Ye, Jiaji Zhang, and Yang Yu. Any-step dynamics model improves future predictions for online and offline reinforcement learning. InThe 13th International Conference on Learning Representations (...

    2025

  20. [28]

    Adm-v2: Pursuing full-horizon roll-out in dynamics models for offline policy learning and evaluation

    Haoxin Lin, Siyuan Xiao, Yi-Chen Li, Zhilong Zhang, Yihao Sun, Chengxing Jia, and Yang Yu. Adm-v2: Pursuing full-horizon roll-out in dynamics models for offline policy learning and evaluation. InThe 14th International Conference on Learning Representations (ICLR’26), Rio de Ja...

    2026

  21. [29]

    From kepler to newton: Inductive biases guide learned world models in transformers.ArXiv, abs/2602.06923, 2026

    Ziming Liu, Sophia Sanborn, Surya Ganguli, and Andreas Tolias. From kepler to newton: Inductive biases guide learned world models in transformers.ArXiv, abs/2602.06923, 2026. URLhttps://api.semanticscholar.org/CorpusID:285401944

    2026

  22. [30]

    Machado, Andre Barreto, Doina Precup, and Michael Bowling

    Marlos C. Machado, Andre Barreto, Doina Precup, and Michael Bowling. Temporal abstraction in reinforcement learning with the successor representation, 2023. URL https://arxiv.org/ abs/2110.05740

    2023 arXiv

  23. [31]

    Transformers are sample-efficient world models

    Vincent Micheli, Eloi Alonso, and François Fleuret. Transformers are sample-efficient world models. InInternational Conference on Learning Representations, 2023

    2023

  24. [32]

    On the difficulty of training recurrent neural networks

    Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. InInternational Conference on Machine Learning, 2012. URL https: //api.semanticscholar.org/CorpusID:14650762. 11

    2012

  25. [33]

    Film: Visual reasoning with a general conditioning layer, 2017

    Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer, 2017. URL https://arxiv.org/abs/ 1709.07871

    2017 arXiv

  26. [34]

    Roformer: Enhanced transformer with rotary position embedding, 2023

    Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, 2023. URL https://arxiv.org/ abs/2104.09864

    2023 arXiv

  27. [35]

    Dyna, an integrated architecture for learning, planning, and reacting.ACM Sigart Bulletin, 2(4):160–163, 1991

    Richard S Sutton. Dyna, an integrated architecture for learning, planning, and reacting.ACM Sigart Bulletin, 2(4):160–163, 1991

    1991

  28. [36]

    Self-correcting models for model-based reinforcement learning, 2017

    Erik Talvitie. Self-correcting models for model-based reinforcement learning, 2017. URL https://arxiv.org/abs/1612.06018

    2017 arXiv

  29. [37]

    dm_control: Software and tasks for continuous control.Software Impacts, 6:100022, 2020

    Saran Tunyasuvunakool, Alistair Muldal, Yotam Doron, Siqi Liu, Steven Bohez, Josh Merel, Tom Erez, Timothy Lillicrap, Nicolas Heess, and Yuval Tassa. dm_control: Software and tasks for continuous control.Software Impacts, 6:100022, 2020. ISSN 2665-9638. doi: https:// doi.org/1...

    2020

  30. [38]

    Chang, Ashesh Rambachan, and Sendhil Mullainathan

    Keyon Vafa, Peter G. Chang, Ashesh Rambachan, and Sendhil Mullainathan. What has a foundation model found? using inductive bias to probe for world models.ArXiv, abs/2507.06952,

  31. [39]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https://arxiv. org/abs/1706.03762

    2023 arXiv

  32. [40]

    Cloning deterministic worlds: The critical role of latent geometry in long-horizon world models, 2026

    Zaishuo Xia, Yukuan Lu, Xinyi Li, Yifan Xu, and Yubei Chen. Cloning deterministic worlds: The critical role of latent geometry in long-horizon world models, 2026. URL https://arxiv. org/abs/2510.26782

    2026 arXiv

  33. [41]

    Leveraging jumpy models for planning and fast learning in robotic domains, 2023

    Jingwei Zhang, Jost Tobias Springenberg, Arunkumar Byravan, Leonard Hasenclever, Abbas Abdolmaleki, Dushyant Rao, Nicolas Heess, and Martin Riedmiller. Leveraging jumpy models for planning and fast learning in robotic domains, 2023. URL https://arxiv.org/abs/ 2302.12617

    2023 arXiv

  34. [42]

    Hierarchical planning with latent world models, 2026

    Wancong Zhang, Basile Terver, Artem Zholus, Soham Chitnis, Harsh Sutaria, Mido Assran, Randall Balestriero, Amir Bar, Adrien Bardes, Yann LeCun, and Nicolas Ballas. Hierarchical planning with latent world models, 2026. URLhttps://arxiv.org/abs/2604.03208

    2026 arXiv

  35. [43]

    Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn

    Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware, 2023. URLhttps://arxiv.org/abs/2304.13705. 12 A Lipschitz Proof of Error Bound We now compare how prediction errors scale with the horizon K under t...

    2023 arXiv

  36. [2013]

    doi: 10.1613/jair.3912

    ISSN 1076-9757. doi: 10.1613/jair.3912. URL http://dx.doi.org/10.1613/jair. 3912

    doi

  37. [2025]

    URLhttps://api.semanticscholar.org/CorpusID:280150828

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.