Pith. sign in

REVIEW 3 major objections 5 minor 25 references

Trajectory World Models for Heterogeneous Environments

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pre-training a world model on 1.3M trajectories from 80 heterogeneous environments transfers to unseen robots, improving prediction, off-policy evaluation, and model predictive control.

desk verdict Solid empirical step forward in world-model pre-training, but the 'heterogeneous transfer' claim is weakened by DMC analogues of the downstream robots in the pre-training data. read the letter →

arxiv 2502.01366 v2 pith:PZXUEWS7 submitted 2025-02-03 cs.LG

Shaofeng Yin , Jialong Wu , Siqiao Huang , Xingjian Su , Xu He , Jianye Hao , Mingsheng Long This is my paper · ORCID
classification cs.LG
keywords worldmodelspre-trainingheterogeneousenvironmentscross-environmenttransferin-contextlearningoff-policyevaluationmodelpredictivecontroltrajectorydatasets
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a world model—a learned predictor of how actions change a system's state—can be pre-trained on trajectories from many different control environments and then transferred to new environments with different sensors, actuators, and dynamics. To make that possible, it curates UniTraj (1.3 million trajectories, 719 million steps, from 80 environments) and builds TrajWorld, a Transformer that reads each scalar state, action, and reward channel as an individual token rather than as a fixed-length vector. After pre-training and fine-tuning, TrajWorld reduces transition-prediction error on the HalfCheetah, Hopper, and Walker2D robots of the D4RL benchmark, sets a new state of the art on the DOPE off-policy evaluation benchmark, and improves online model predictive control. The payoff if true is practical: a single pre-trained model carries general knowledge of physical dynamics, so a new robot can get an accurate predictive model with far fewer costly interactions.

What carries the argument

The mechanism is scalar-level tokenization plus interleaved two-axis attention. TrajWorld flattens a trajectory into a two-dimensional matrix of timesteps by variates, where each variate is a single scalar dimension of the state, action, or reward; every scalar is discretized into one of $B$ uniform bins, and the resulting categorical tokens are embedded with learned timestep, variate, and prediction-target embeddings. Each transformer block then alternates causal attention along the time axis (per variate) with unmasked attention along the variate axis (per timestep), so any predicted quantity can aggregate information from all variates at all previous timesteps. The model never receives an environment ID: it must identify which environment it is in, and how actions map to state changes, from the recent trajectory history, which the authors call capturing environment dynamics in-context. Predictions are produced as a next-step categorical distribution over the discretized state and reward variates, trained with cross-entropy, so all variates are predicted jointly rather than one after another.

What would settle it

Re-train TrajWorld on UniTraj after deleting every pre-training environment that resembles the test robots—all DMC walker, cheetah, and hopper bodies, plus their Modular RL and TD-MPC2 variants—and then fine-tune on the Gym HalfCheetah, Hopper, and Walker2D D4RL datasets; if the gains over from-scratch training largely vanish, the claimed heterogeneous transfer is mostly near-duplicate morphology recall, while persistent gains would support the paper's interpretation. A complementary check is to rerun the zero-shot Cart-2-Pole and Cart-3-Pole rollouts with history context removed and confirm that prediction error rises, which would verify that in-context inference, not memorized per-environment statistics, carries the transfer.

Watch

Extended reading notes

Core claim

The paper's central claim is that low-dimensional sensor readings, despite looking different across robots, contain enough shared structure for large-scale pre-training—provided the model treats every scalar sensor or actuator channel as a token and infers each environment's dynamics from its own trajectory history, with no environment identifier given. Concretely, the authors report that pre-training TrajWorld on UniTraj and then fine-tuning it on downstream data yields substantial gains in transition prediction, a new state of the art for off-policy evaluation, and superior online performance of model predictive control, and they describe this as the first demonstrated transfer of world models across heterogeneous, complex control environments. The claim is carried jointly by the data and the architecture: UniTraj is scaled while keeping diversity across data sources, policy levels, and 80 embodiments, and TrajWorld's interleaved temporal and variate attention lets it identify how actions drive state changes from context, which is what the zero-shot experiments on cart-like environments are meant to show.

Load-bearing premise

The transfer claim depends on the three test robots being genuinely unseen to the model, even though very similar versions of those same robot bodies were included in its pre-training data, and the paper assumes the differences in state definition, action dimension, and dynamics make the transfer non-trivial.

Editorial extensions

If this is right

  • A newly encountered environment needs far fewer collected trajectories: fine-tuning the pre-trained model beats training from scratch on the same data, and the gap widens as data grows scarcer.
  • Off-policy evaluation—scoring candidate policies by rolling them out in the learned model—becomes accurate enough to set a new state of the art on the DOPE benchmark, which makes offline policy selection cheaper and safer.
  • Model predictive control using the pre-trained world model improves online returns on fragile robots like Hopper and Walker2D, because more accurate predictions keep planned actions away from unsafe states.
  • The dataset-scale ablations show a consistent scaling trend: larger and more diverse pre-training data yields better downstream prediction and control, so the gains are expected to grow if UniTraj is extended.
  • Architecture choice is part of the result: the same pre-training data hurts an MLP ensemble (negative transfer) and transfers less through the one-dimensional sequence baseline TDM, showing that the two-axis attention design is load-bearing rather than incidental.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's strongest evidence for true heterogeneity would be a pre-training set with every near-relative of the test robots removed; since DMC versions of walker, cheetah, and hopper bodies appear in UniTraj, part of the reported gain could reflect near-duplicate morphology, and an exclusion study would settle how much.
  • If the in-context mechanism is general, TrajWorld should handle entirely new sensor channels (touch, IMU, camera-derived features) as just new variate tokens, a testable prediction beyond anything the paper runs.
  • The learned continuity of bin orderings suggests a direct fix for the paper's acknowledged fixed-range limitation: extending bin boundaries beyond the observed data range should let the discretized model extrapolate like a regression model, which is a concrete experiment the authors flag but do not run.
  • A scalar-as-token world model points toward a shared dynamics prior across simulation, robotics, and teleoperation data, where the same pre-trained model seeds many downstream task-specific models; the authors gesture at this direction with multimodal vision-plus-proprioception pre-training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces UniTraj, a dataset of 1.3M trajectories from 80 control environments with varying state/action dimensions, and TrajWorld, a transformer that scalarizes each state/action/reward component and applies interleaved temporal and variate attention. TrajWorld is pre-trained on UniTraj and then fine-tuned on D4RL datasets of Gym HalfCheetah, Hopper, and Walker2D. The authors report that pre-training reduces transition-prediction MAE, improves off-policy evaluation (OPE) on the DOPE benchmark, and improves model-predictive-control (MPC) returns, with qualitative zero-shot predictions on Cart-2-Pole/Cart-3-Pole. They claim the first successful transfer of world models across heterogeneous complex control environments.

Significance. The dataset and architecture are useful contributions, and the experimental protocol is extensive (75 train-test pairs, three seeds, multiple baselines, scale/diversity ablations). If the transfer results hold for genuinely unseen environments, this would be a notable step toward generalist world models for low-dimensional continuous control. The central caveat is that the downstream Gym environments are not established to be genuinely unseen, because DMC analogues of the same robots appear in UniTraj and the appendix ablation does not remove that overlap. The zero-shot evidence is qualitative and concerns morphology-near environments. The OPE state-of-the-art claim is stronger than the per-task results. These issues affect the headline claim and need to be addressed before publication.

major comments (3)
  1. [§5.2, Table 2, Appendix A.2 and C.7] The central claim that fine-tuning targets are "previously unseen" environments is not established. UniTraj contains DMC analogues of the same robots (Walker, Hopper, Cheetah) in ExORL, RL Unplugged, DB-1, and TD-MPC2; the paper excludes only the OpenAI Gym versions (Section 3) and the ablation in Appendix C.7 removes only TD-MPC2 and Modular RL, so these DMC analogues remain in the pre-training set. Because the DMC and Gym versions share the same body morphology and state/action semantics (joint angles, velocities, torques), the reported gains may largely reflect near-duplicate morphology rather than general cross-environment transfer. The Table 2 note that Gym and DMC versions differ in state/action definitions and parameters is an assumption, not a control. Please re-run the main transition-prediction, OPE, and MPC experiments with a pre-training set that excludes all DMC/Modular-RL analogues of the three test robots, or otherwise demonstrate that the gains persist on environments with non-overlapping morphology.
  2. [§5.1, Figures 4b, 13, 14] The zero-shot cross-environment evidence is qualitative and restricted to Cart-2-Pole/Cart-3-Pole, which are morphologically very close to Cartpole environments present in UniTraj. Visual inspection of a small number of trajectories does not substantiate the claim of zero-shot transfer to arbitrary unseen sensor/actuator sets. Please provide quantitative prediction errors on a held-out set of test trajectories for these environments, and preferably add evaluation on a held-out embodiment that is more distant from the training morphologies.
  3. [§5.3, Table 6] The abstract's phrase "achieves a new state-of-the-art for off-policy evaluation" is stronger than the reported results. In Table 6, TrajWorld (w/ PT) is worse than ETM on several environment-level tasks (e.g., Hopper-medium raw absolute error 127 vs. 47, Hopper-medium-replay 73 vs. 29, HalfCheetah-random 1059 vs. 842) and it underperforms on Regret@1 as acknowledged in the text. The SOTA claim should be qualified to the averaged normalized absolute error and rank-correlation metrics, and the per-environment results should be discussed in the main text.
minor comments (5)
  1. [§5.4] The sentence "evaluting both both from-scratch and fine-tuned variants" contains a duplicated "both" and a typo in "evaluting".
  2. [§6] In the Related Work section, "no for computational efficiency" appears to be a typo for "not only for computational efficiency".
  3. [Contributions bullet, §1] The bullet "resulting in simultaneous and significant improvements in transition prediction, off-policy policy, and model predictive control" should say "off-policy evaluation" instead of "off-policy policy".
  4. [Figure 1 caption] The caption reads "Y-axis at log scale"; it should be "Y-axis on a log scale".
  5. [Appendix B.1] The text "two Gaussian distributions with the same standard derivation" should be "standard deviation".

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the transition-prediction, OPE, and MPC gains are measured on held-out external benchmarks after fine-tuning, with no fitted parameter renamed as a prediction.

full rationale

I walked the derivation chain from the UniTraj pre-training corpus through the TrajWorld architecture to the downstream evaluations. The world model is pre-trained on UniTraj, then fine-tuned on each D4RL dataset (HalfCheetah, Hopper, Walker2D) and evaluated on held-out test sets from the same D4RL environments, on the DOPE off-policy-evaluation benchmark, and in online MPC. None of these quantities is produced by a fitted parameter that was tuned to the reported target: the pre-training provides a fixed initial checkpoint and the downstream metrics are empirical errors from rollouts in the environments. The comparison is also made against from-scratch training of the same architecture and against external baselines (MLP Ensemble, TDM, ETM), so the claimed gains are not implied by the model definition. The architecture choices (scalarization, interleaved temporal/variate attention, next-step cross-entropy) are stated as design decisions whose value is established empirically in Section 5, not imported from a self-citation chain. The only self-citations (Wu et al., 2024a,b; Wu et al., 2025) appear in the related-work survey of video/text world-model pre-training and are not load-bearing for the paper's claims. The manuscript also flags genuine limitations (bounded discretization range, calibration difficulty, computational cost) that are unrelated to circular reasoning. The strongest validity concern is that DMC analogues of the downstream robots appear inside UniTraj while the paper calls the downstream environments 'previously unseen' (Section 3 and Table 2); however, this is a data-overlap/external-validity question, not a case where the result is equivalent to its inputs by construction, so it does not raise the circularity score under the stated rubric.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. Its contributions are an architecture and a dataset. The central claims depend on design assumptions about scalar homogeneity, in-context learning, and discretization adequacy, none of which are formally proven but are supported by empirical results.

free parameters (4)
  • Sampling weights for UniTraj subsets = ExoRL 75, RLU 5, JAT 90, DB-1 1, TD-MPC2 90, Modular RL 30
    Manually chosen weights (Table 3) balance dataset components during pre-training; they affect the learned distribution and downstream transfer.
  • Number of discretization bins B = Not stated
    The resolution of the categorical representation is a hyperparameter that controls prediction granularity and bounded range.
  • MPC noise standard deviation = Hopper 0.05, Walker2D 0.2, HalfCheetah 0.025
    Tuned individually per environment in Appendix B.5, though applied consistently across models for fair comparison.
  • Transformer context length = 20
    History context window for in-context dynamics inference; a hyperparameter that may affect transfer.
assumptions (3)
  • domain assumption Scalar homogeneity: each scalar variate can be modeled consistently across environments regardless of vector shape.
    Section 4.1 argues that each state/action dimension is a fundamental quantity and can be processed by a shared scalar-level model. This is a design assumption that enables the architecture.
  • domain assumption In-context environment identification: historical transitions provide enough context to infer environment dynamics without explicit environment IDs.
    Section 4.1 states that the model relies on in-context learning rather than environment IDs. This assumes the Transformer can identify dynamics from a 20-step history, which is not guaranteed for all environments.
  • domain assumption Sufficient expressiveness of uniform bin discretization with bin centers.
    Section 4.2 discretizes scalars into B bins with boundaries from training data. This bounds predictions and assumes the bin resolution is fine enough for accurate transition modeling, an acknowledged limitation in Appendix D.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Trajectory World Models for Heterogeneous Environments." pith.science (2026). https://pith.science/paper/PZXUEWS7

@misc{pith2026250201366,
  author       = {Pith},
  title        = {Pith review of: Trajectory World Models for Heterogeneous Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PZXUEWS7}},
  note         = {Machine review of arXiv:2502.01366}
}
read the original abstract

Heterogeneity in sensors and actuators across environments poses a significant challenge to building large-scale pre-trained world models on top of this low-dimensional sensor information. In this work, we explore pre-training world models for heterogeneous environments by addressing key transfer barriers in both data diversity and model flexibility. We introduce UniTraj, a unified dataset comprising over one million trajectories from 80 environments, designed to scale data while preserving critical diversity. Additionally, we propose TrajWorld, a novel architecture capable of flexibly handling varying sensor and actuator information and capturing environment dynamics in-context. Pre-training TrajWorld on UniTraj yields substantial gains in transition prediction, achieves a new state-of-the-art for off-policy evaluation, and also delivers superior online performance of model predictive control. To the best of our knowledge, this work, for the first time, demonstrates the transfer benefits of world models across heterogeneous and complex control environments. Code and data are available at https://github.com/thuml/TrajWorld.

Figures

Figures reproduced from arXiv: 2502.01366 by the authors.

Figure 1
Figure 1. Aggregated transition prediction error (MAE) across 75 train-test dataset pairs, comparing MLP Ensemble (Chua et al., 2018), TDM (Schubert et al., 2023), and proposed TrajWorld, with and without pre-training on UniTraj dataset. Y-axis at log scale. existing methods often learn world models tabula rasa, rely￾ing on data from a single, specific environment. This limits their ability to generalize to out-of-distributio… view at source ↗
Figure 2
Figure 2. Illustration of pre-training a world model from heterogeneous environments, with each environment labeled by its state and action dimensions. A Trajectory World Model, designed for flexibility in handling divergent state and action definitions, demonstrates effective positive transfer across distinct, heterogeneous, and complex control environments. to bridge this gap by exploring the potential of pre-training a wor… view at source ↗
Figure 3
Figure 3. Architecture of Trajectory World Models. A trajectory is first flattened into scalars, organized into two dimensions by timesteps and variates (each variate corresponds to a single dimension in the state, action, and reward), and then discretized into categorical representations. A Transformer with interleaved temporal and variate attentions processes the inputs to predict the categorical distribution for the next t… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Zero-shot generalization. (a) Mean squared error of zero￾shot transition predictions in modified Gym Pendulum (holdout gravity) and Walker2D (holdout friction etc.). (b) TrajWorld’s zero-shot predictions for two Cart-2-Pole trajectories, which share 10 context steps bu…
Figure 5
Figure 5. Figure 5: Mean absolute errors (MAE) of transition prediction for TrajWorld, with and without pre-training (PT), across different train-test dataset pairs. Each subplot corresponds to a distinct training dataset, with the test datasets shown on the x-axis (r=random, m-r=medium￾r…
Figure 6
Figure 6. Figure 6: Overall off-policy evaluation (OPE) results across 15 datasets of 3 environments, averaged across three random seeds. jWorld achieves a lower average prediction error across 5 test datasets, further validating the effectiveness of pre￾training. Moreover, the transfer b…
Figure 7
Figure 7. Figure 7: Model predictive control (MPC) results with proposal policies across three environments, averaged over three random seeds. the TDM model exhibits negative transfer in the MPC with proposal policies setting, despite showing positive transfer in transition prediction and…
Figure 8
Figure 8. Figure 8: Model analysis. (a) Downstream prediction error of TrajWorld under varying data scarcity levels. (b) t-SNE visualization of the linear weights in the model’s prediction head. (c) Variate attention map from the third layer of TrajWorld fine-tuned on Walker2D. None 1% 10…
Figure 9
Figure 9. Figure 9: Effects of pre-training scale and diversity. (a) Aggregated transition prediction error. (b) Model predictive control performance on Walker2D and Hopper. All results are obtained from models fine-tuned from pre-trained TrajWorld on different subsets of UniTraj. ments w…
Figure 10
Figure 10. Figure 10: Mean absolute errors (MAE) of transition prediction for MLP Ensemble, with and without pre-training (PT), across different train-test dataset pairs. Each subplot corresponds to a distinct training dataset, with the test datasets shown on the x-axis (r=random, m-r=medi…
Figure 11
Figure 11. Figure 11: Mean absolute errors (MAE) of transition prediction for TDM, with and without pre-training (PT), across different train-test dataset pairs. Each subplot corresponds to a distinct training dataset, with the test datasets shown on the x-axis (r=random, m-r=medium￾replay…
Figure 12
Figure 12. Figure 12: Model predictive control (MPC) results using a random shooting planner, averaged across three random seeds. The proposal policy line indicates the performance of a random action-sampling strategy. C.4. Additional Model Predictive Control Results Quantitative results w…
Figure 13
Figure 13. Figure 13: TrajWorld’s zero-shot predictions for two Cart-3-Pole trajectories, which share 10 context steps but diverge due to differing subsequent actions. Comparison with baselines. We also provide zero-shot prediction from other baselines in [PITH_FULL_IMAGE:figures/full_fig…
Figure 14
Figure 14. Figure 14: Zero-shot predictions from different pre-trained models on two Cart-2-Pole trajectories that share the same 10 context steps but diverge thereafter due to different future actions. C.6. Additional Variate Attention Visualization We present the variate attention maps o…
Figure 15
Figure 15. Figure 15: Variate attention maps of our pre-trained TrajWorld Model, fine-tuned under Walker2D environment [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: Variate attention maps of our TrajWorld Model in the Walker2D environment, trained from scratch. C.7. Additional Ablation Study on Pre-training Dataset To investigate the contributions of different components of the UniTraj dataset to the pre-training process, we cond…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 6 canonical work pages

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [4]

    Baseline: MLP Ensemble

    Hyperparameters for TrajWorld. Baseline: MLP Ensemble. Following prior work (Chua et al., 2018; Janner et al., 2019; Yu et al., 2020b), we train an ensemble of transition models, parameterized as a diagonal Gaussian distribution of the next state and reward, implemented using MLPs. These models are trained with bootstrapped training samples, and optimized...

    work page 2018

  3. [7]

    Jack of all trades, master of some, a multi-purpose trans- former agent

    Gallou´edec, Q., Beeching, E., Romac, C., and Dellandr´ea, E. Jack of all trades, master of some, a multi-purpose trans- former agent. arXiv preprint arXiv:2402.09844,

    arXiv

  4. [9]

    Axial attention in multidimensional transformers

    Ho, J., Kalchbrenner, N., Weissenborn, D., and Salimans, T. Axial attention in multidimensional transformers. arXiv preprint arXiv:1912.12180,

    arXiv 1912

  5. [10]

    and Nachum, O

    Kostrikov, I. and Nachum, O. Statistical bootstrapping for uncertainty estimation in off-policy evaluation. arXiv preprint arXiv:2007.13609,

    arXiv 2007

  6. [12]

    T., Byravan, A., Hasen- clever, L., and Heess, N

    Schubert, I., Zhang, J., Bruce, J., Bechtle, S., Parisotto, E., Riedmiller, M., Springenberg, J. T., Byravan, A., Hasen- clever, L., and Heess, N. A generalist dynamics model for control. arXiv preprint arXiv:2305.10912,

    arXiv

  7. [13]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347 ,

    arXiv

  8. [14]

    Deepmind control suite

    Tassa, Y ., Doron, Y ., Muldal, A., Erez, T., Li, Y ., de Las Casas, D., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., Lillicrap, T., and Riedmiller, M. Deepmind control suite. arXiv preprint arXiv: 1801.00690 ,

    arXiv

Show all 25 references
  1. [15]

    Team, O. E. L., Stooke, A., Mahajan, A., Barros, C., Deck, C., Bauer, J., Sygnowski, J., Trebacz, M., Jaderberg, M., Mathieu, M., et al. Open-ended learning leads to gener- ally capable agents. arXiv preprint arXiv:2107.12808 ,

    arXiv

  2. [16]

    On realiza- tion of intelligent decision-making in the real world: A foundation decision model perspective

    Wen, Y ., Wan, Z., Zhou, M., Hou, S., Cao, Z., Le, C., Chen, J., Tian, Z., Zhang, W., and Wang, J. On realiza- tion of intelligent decision-making in the real world: A foundation decision model perspective. arXiv preprint arXiv:2212.12669,

    arXiv

  3. [17]

    Pre-training con- textualized world models with in-the-wild videos for re- inforcement learning

    Wu, J., Ma, H., Deng, C., and Long, M. Pre-training con- textualized world models with in-the-wild videos for re- inforcement learning. In NeurIPS, 2024a. Wu, J., Yin, S., Feng, N., He, X., Li, D., Hao, J., and Long, M. ivideogpt: Interactive videogpts are scalable world model...

  4. [18]

    Don’t change the algorithm, change the data: Exploratory data for offline reinforce- ment learning

    Yarats, D., Brandfonbrener, D., Liu, H., Laskin, M., Abbeel, P., Lazaric, A., and Pinto, L. Don’t change the algorithm, change the data: Exploratory data for offline reinforce- ment learning. arXiv preprint arXiv:2201.13425,

    arXiv

  5. [19]

    Y ., et al

    Ye, S., Jang, J., Jeon, B., Joo, S., Yang, J., Peng, B., Mandlekar, A., Tan, R., Chao, Y .-W., Lin, B. Y ., et al. Latent action pretraining from videos. arXiv preprint arXiv:2410.11758,

    arXiv

  6. [21]

    UniTraj Dataset Details A.1

    12 Trajectory World Models for Heterogeneous Environments A. UniTraj Dataset Details A.1. Overview of UniTraj Components In this part, we provide a brief overview of each component of the UniTraj dataset. ExORL (Yarats et al., 2022). Exploratory Data for Offline RL (ExORL) fol...

    2020

  7. [24]

    DICE (Yang et al., 2020), and Variational Power Method (VPM) (Wen et al., 2020). B.4.3. M ETRICS We adopt the evaluation metrics used in the DOPE benchmark. Mean Absolute Error. The absolute error quantifies the deviation between the true value and the estimated value of a pol...

    2020

  8. [25]

    OPE results for a four-layer TrajWorld model trained from scratch compared to a model fine-tuned from a pre-trained version on the ablation dataset, averaged over two seeds. D. Extended Discussion Limitations of bounded prediction. Our discretization scheme (Section 4.2) has t...

    2023

  9. [1000]

    We use KV cache to accelerate the rollouts of our TrajWorld. B.4.2. B ASELINES We primarily compare against model-based OPE with Energy-based Transition Models (ETM) (Chen et al., 2024), a strong baseline that significantly outperforms previous methods and represents state-of-...

    2024

  10. [2018]

    Brockman, G

    URL http://github.com/jax-ml/jax. Brockman, G. Openai gym. arXiv preprint arXiv:1606.01540,

    arXiv

  11. [2019]

    A path towards autonomous machine intelligence version 0.9

    LeCun, Y . A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62(1):1–62,

    2022

  12. [2020]

    Visual foresight: Model-based deep reinforcement learning for vision-based robotic control

    Ebert, F., Finn, C., Dasari, S., Xie, A., Lee, A., and Levine, S. Visual foresight: Model-based deep reinforcement learning for vision-based robotic control. arXiv preprint arXiv:1812.00568,

    arXiv

  13. [2021]

    Dino-wm: World models on pre-trained visual features enable zero- shot planning

    Zhou, G., Pan, H., LeCun, Y ., and Pinto, L. Dino-wm: World models on pre-trained visual features enable zero- shot planning. arXiv preprint arXiv:2411.04983,

    arXiv

  14. [2022]

    D4rl: Datasets for deep data-driven reinforcement learning

    Fu, J., Kumar, A., Nachum, O., Tucker, G., and Levine, S. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219,

    2004 arXiv

  15. [2023]

    Cosmos world foundation model platform for physical ai

    Agarwal, N., Ali, A., Bala, M., Balaji, Y ., Barker, E., Cai, T., Chattopadhyay, P., Chen, Y ., Cui, Y ., Ding, Y ., et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575,

    arXiv

  16. [2024]

    Is your llm secretly a world model of the internet? model-based plan- ning for web agents

    Gu, Y ., Zheng, B., Gou, B., Zhang, K., Chang, C., Srivas- tava, S., Xie, Y ., Qi, P., Sun, H., and Su, Y . Is your llm secretly a world model of the internet? model-based plan- ning for web agents. arXiv preprint arXiv:2411.06559,

    arXiv

  17. [2025]

    Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation

    Cheang, C.-L., Chen, G., Jing, Y ., Kong, T., Li, H., Li, Y ., Liu, Y ., Wu, H., Xu, J., Yang, Y ., et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158,

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.