Pith. sign in

REVIEW 3 major objections 5 minor 29 references

Is Forward Prediction Enough? Physical State Grounding for JEPA World Models

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Physical state grounding beats forward-only JEPA world models

desk verdict A solid empirical paper whose downstream gains are probably real, but whose identifiability diagnosis is partly circular and whose causal claim lacks a matched control. read the letter →

arxiv 2608.06799 v1 pith:ZU4NBCR2 submitted 2026-08-07 cs.RO cs.CV

Haodong Yan , Jiaguan Zhu , Mingyuan Jia , Ruiqing Yin , Junjie He , Zhide Zhong , Junfeng Li , Jinxuan Lu
show 7 more authors
Hengtao Li Tianran Zhang Jiayi Chen Wenxuan Song Wen Chen Yuxiang Gao Haoang Li
This is my paper · ORCID
classification cs.ROcs.CV
keywords physicalstategroundingJEPAworldmodelslatentidentifiabilitygoal-conditionedplanningpolicylearningproprioceptionjoint-anglechangesrobotmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that forward prediction alone is not enough for a JEPA world model to serve robot control, because the learned latents do not reliably expose physical state or state changes. It proposes PSG-JEPA, which adds two training-only grounding objectives: a state head that predicts robot proprioception from each latent, and a transition head that predicts multi-horizon joint-angle changes from latent pairs. In probe-based identifiability, goal-conditioned planning on frozen latents, and policy learning in simulation and on a real robot, PSG-JEPA consistently beats forward-only, action-supervised, and frozen-feature baselines. The inference architecture and computational cost remain unchanged because the grounding heads are discarded after training.

What carries the argument

The argument rides on two lightweight heads attached to the shared JEPA latent during training. The state head $H_s$ regresses each latent $z_t$ to the aligned proprioceptive state $s_t$, making physical state readable from a single latent; the transition head $H_\Delta$ regresses every endpoint pair $(z_{t+i}, z_{t+i+k})$ to the joint-angle change $\Delta q_{t+i,k}=q_{t+i+k}-q_{t+i}$ for all horizons $k=1,\dots,T-1$, with each horizon weighted equally so short-range pairs do not dominate. The total objective is $L_{PSG}=L_{JEPA}+\lambda_g(L_{static}+L_{dynamic})$, where $L_{JEPA}$ retains forward prediction and the SIGReg anti-collapse term. Because $\Delta q$ is uniquely determined by endpoint states and fixed-dimensional across horizons, it avoids the ambiguity of inverse-dynamics action supervision.

What would settle it

On OGBench-Cube, take the same PSG-JEPA training pipeline and shift the logged joint-angle stream by 2–5 control steps relative to the images, then rerun the single-latent/transition probes and the 5-epoch GC-IDM planning evaluation. If the identifiability gains and fast-planning lead over LeWM disappear or reverse, the result depends on the assumed alignment rather than on the grounding targets per se.

Watch

Extended reading notes

Core claim

The paper's central claim is that forward-trained JEPA representations suffer a robot-centric identifiability gap: single latents fail to decode important proprioceptive variables such as end-effector yaw, and latent pairs only partially decode physical transitions. PSG-JEPA closes this gap by supervising the shared planning latent with proprioceptive state and with fixed-dimensional joint-angle changes across all horizons in the training window. As a result, the same backbone that underperforms at 5-epoch planning (LeWM, 80.7% success) reaches 95.0% when trained with grounding, and the grounded representation also lifts LIBERO-Goal policy success from 77.7% to 85.3% and real-robot average success from 60.0% to 79.3%. The grounding heads are removed at inference, so the improvement comes at zero added deployment cost.

Load-bearing premise

The paper assumes the logged robot state is temporally aligned with the camera frames used to compute latents, but it never describes the alignment procedure or quantifies synchronization error; if the streams are off by even a few control steps, the grounding targets become noisy and the claimed gains could dissolve.

Editorial extensions

If this is right

  • Planners on grounded latents need far less data and optimization: at 5 GC-IDM epochs PSG-JEPA already scores 95.0% on OGBench-Cube versus 80.7% for LeWM, and at 25% demonstration data it keeps a double-digit lead.
  • The same grounded representation helps task-conditioned policies, not just planning: +7.6 points on LIBERO-Goal and +19.3 points average on three real-robot tasks over the LeWM baseline.
  • Grounding does not degrade the forward model: recursive latent prediction error falls at every horizon, with a 67% lower MSE at the longest rollout on OGBench-Cube.
  • Because the grounding heads are discarded after training, PSG-JEPA is a drop-in training-time change to LeWM-style JEPA world models with no inference-time overhead.
  • Multi-horizon transition grounding contributes beyond adjacent-pair supervision: replacing it with adjacent-only or endpoint-only horizons drops policy success from 85.3% to about 81% and planning from 95.0% to about 93.5%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The identifiability-gap diagnosis suggests that forward-prediction objectives are not state-anchored; any cheap, temporally aligned physical measurement (Cartesian pose, object state, contact forces) could be swapped into the same grounding recipe, a testable extension for other robot domains.
  • Because the grounding heads are disposable, the same forward backbone could be re-grounded for a new robot or task with a short fine-tuning pass, offering a cheap domain-adaptation route for latent world models.
  • The results implicitly hinge on state–image temporal alignment. A direct test would deliberately misalign the logged joint-angle stream by a few control steps and check whether the identifiability and planning gains vanish; the paper neither describes its alignment procedure nor reports such a stress test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper asks whether action-conditioned forward prediction in JEPA world models is sufficient for robot control, and argues that it is not: forward-only training leaves robot-centric physical state poorly identifiable from latents. The authors propose PSG-JEPA, which adds two training-only grounding losses to the LeWM backbone: a static loss regressing each latent to logged proprioceptive state (Eq. 4) and a dynamic loss regressing each latent pair to multi-horizon joint-angle changes (Eq. 6). The grounding heads are discarded at inference. Evaluation is at three levels: probe-based identifiability (Tables 1-2), goal-conditioned planning on frozen latents with the GC-IDM planner (Table 3, Fig. 3), and policy learning in LIBERO-Goal and on a real dual-arm robot (Table 4, Fig. 4). Across these levels, PSG-JEPA reports higher scores than LeWM, LeWMActionIDM, and DINOv2, and ablations (Table 5) indicate that both grounding terms contribute.

Significance. If the causal claim is accepted, the paper makes a practically useful contribution: a simple, inference-free modification to JEPA world-model training improves downstream planning and policy performance, and the evaluation is unusually thorough for this area, spanning probes, planning, simulation policy learning, and real-robot manipulation. The same-backbone comparison to LeWM and the three-seed reporting in simulation are strengths. The main caveat is that the Q1 identifiability results are partly by construction, and no baseline controls for a matched non-physical auxiliary objective, so the specifically physical nature of the benefit is not yet isolated. With that isolation, the result would be significant; as it stands, it is a promising but incompletely supported causal claim.

major comments (3)
  1. [Section 4.3, Tables 1-2; Eq. (4), Eq. (6)] The Q1 probe evaluation partly re-measures the training objectives, so it does not independently establish an identifiability gap. Table 1 probes predict exactly the components of s_t used as targets in Eq. (4), and the JointVel row of Table 2 is exactly the k=1 target of Eq. (6). High probe r for PSG-JEPA therefore reflects the existence of the trained decoding heads H_s and H_Delta. The comparison to LeWM shows that supervised targets help, but it does not show that the physical content of those targets, rather than the mere addition of regression supervision or auxiliary heads, is what creates the gain. Please add a matched auxiliary-objective control (e.g., a head regressing a scrambled or non-physical target with identical architecture and loss weight) and, in the absence of such a control, soften the identifiability-gap claim.
  2. [Section 4.4-4.5, Tables 3-4] The central causal claim that physical grounding, rather than extra regression supervision in general, drives the planning and policy gains is not isolated. LeWMActionIDM adds an action-prediction target but uses no state head and a different target, and LeWM has no auxiliary head; neither is a matched control for the architecture or for the amount of additional supervision. Please include a control with the same head architecture and training budget but a non-physical target (or a permutation or scrambling of the physical targets) to show that the physical meaning of the targets is what matters. Without this, the title's claim that forward prediction is insufficient is not yet fully supported.
  3. [Figure 2 and Section 4.5, Real-World Robot] The real-robot results are reported without variance or replication: Fig. 4 gives one success rate per task from a single run of 50 trials, and no information is provided about the number of policy-training seeds or runs. Please report multiple seeds or at least trial-level confidence intervals, and describe how logged robot state is synchronized with camera frames. Figure 2 only calls the state 'aligned' without specifying the alignment procedure, the timestamp source, or the expected synchronization error, which matters because Eq. (4) and Eq. (6) pair each latent with the state at that frame.
minor comments (5)
  1. [Figure 3] The x-axis label '1 5 25 100' does not match the epoch values in Table 3 (5, 10, 25, 100); please check the axis values and use consistent notation.
  2. [Section 4.4, Open-loop latent prediction] The open-loop prediction results are reported only as text in the paragraph without a table or figure; consider adding a small table or plot for reproducibility.
  3. [Section 3.2, Eq. (4)] The dimension d_s of the proprioceptive state is not defined before Eq. (4); please define d_s explicitly alongside d_q.
  4. [Section 4.5, OFT policy head] The reference for the OFT action head, 'Li et al. (2026), Predictive but not plannable: RC-aux for latent world models,' seems mismatched with the OFT method; please verify and correct the citation.
  5. [Table 5] The 'state' and 'transition' probe rows report means of linear Pearson r across heterogeneous quantities; please state explicitly which quantities are averaged and consider reporting per-quantity values, since averaging correlations over different scales can be misleading.

Circularity Check

2 steps flagged · score 4.0 of 10

The Q1 identifiability probes directly measure the same targets that the grounding losses supervise, so part of the claimed identifiability gain is by construction; Q2 and Q3 are external benchmarks and keep the central claim independently grounded.

  1. fitted input called prediction [Eq. (4) and Sec. 4.3, Table 1 (single-latent state probes)]
    "Lstatic = 1/T ∑_{i=0}^{T-1} MSE(Hs(zt+i), st+i), where st denotes the robot proprioceptive state... Following LeWM, we freeze the encoder and fit linear ridge and shallow MLP probes... Single-latent probes predict robot proprioception from zt. We report Pearson r on held-out test episodes."

    The single-latent probe regresses z_t to s_t, which is exactly the map that the static grounding head H_s was trained to implement in Eq. (4). Up to probe capacity and generalization, a converged static-grounding solution already contains a decoder path for these targets, so high probe r is expected by construction. The gain over LeWM therefore shows that adding a supervised auxiliary objective makes its own targets decodable; it does not independently establish that forward-only training leaves a robot-centric identifiability gap. The probe is an honest check that the objective worked, but as evidence for the method's conceptual claim it is partly self-confirming because the evaluation target and the training target coincide.

  2. fitted input called prediction [Eq. (6) and Sec. 4.3, Table 2 (transition probes)]
    "Ldynamic = 1/|K| ∑_{k∈K} 1/|Ik| ∑_{i∈Ik} MSE(Δq̂t+i,k, Δqt+i,k), where Δqt+i,k = qt+i+k − qt+i... Pairwise probes predict joint velocity, gripper velocity, or action from (zt, zt+1)."

    Joint velocity is q_{t+1} − q_t, exactly the k=1 target of the dynamic grounding loss in Eq. (6). A trained transition head H_Δ already maps every adjacent latent pair to this quantity, so a pairwise probe fitting the same target inherits the learned readout. The reported JointVel gain is therefore partly forced by the training objective rather than independent evidence that latent pairs encode physical change. The gripper-velocity and action probes are not direct dynamic-grounding targets, so they provide some independent content; nevertheless, the headline transition-identifiability result is substantially a measure of the supervised objective itself.

full rationale

No load-bearing self-citation chain appears in the paper: the backbone, SIGReg regularizer, planner, and benchmarks are external works (LeWM, Balestriero and LeCun, Nguyen et al., Park et al., Liu et al.), and the authors' own prior papers are not invoked to justify the method or to forbid alternatives. The central claim of consistent outperformance is supported mainly by Q2 and Q3, which are external: Q2 uses the same GC-IDM planner on frozen latents from all representations, and Q3 uses LIBERO-Goal and a real-robot comparison against LeWM under the same policy-learning protocol. These results do not reduce to the grounding losses, and the open-loop latent MSE comparison is an additional empirical check. The circularity is confined to Q1: probe-based identifiability is, by design, the objective being optimized, so the magnitude of the Q1 gain is expected and cannot by itself establish that physical grounding, as opposed to matched auxiliary regression supervision or architecture changes, drives the downstream gains. LeWMActionIDM is not a matched non-physical auxiliary-objective control with the same head count and target dimensionality, so the causal role of the physical content of the targets remains partly open. The reader's concern about temporal alignment between logged robot state and camera observations is a data-quality and experimental-validity risk, not a circularity, and is not scored here. Because two of the three evaluation levels are external and the method's inference architecture is unchanged, the central claim still has independent content; score 4 reflects the partial construction in the identifiability level without a self-citation chain.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The state and transition heads are architectural components discarded after training.

free parameters (3)
  • lambda_reg (SIGReg weight) = 0.09
    Weight for the anti-collapse regularizer from LeWM; chosen by hand and fixed across all benchmarks (Section 3.1).
  • lambda_g (grounding weight) = 0.1
    Shared weight for static and dynamic grounding losses; chosen by hand and fixed across all benchmarks (Section 3.4).
  • context frames C and window size T = C=3, T=4
    Training window configuration; design choice, not swept (Section 4.1).
assumptions (4)
  • domain assumption Proprioceptive state is available and free in robot training datasets.
    The method relies on logged joint angles and end-effector pose for state grounding; discussed in Section 2.3.
  • standard math The net joint-angle change between two endpoint states is uniquely determined by those states.
    Section 3.3 argues this is why latent pairs can be supervised by joint-angle changes; it is a kinematic fact.
  • domain assumption The SIGReg anti-collapse regularizer is effective at preventing latent collapse in this setting.
    Section 3.1 retains it without modification; if it fails, the latent space could collapse.
  • domain assumption Image observations and proprioceptive state logs are synchronized in the training data.
    Figure 2 and Section 3.2 assume 'aligned logged robot state'.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Is Forward Prediction Enough? Physical State Grounding for JEPA World Models." pith.science (2026). https://pith.science/paper/ZU4NBCR2

@misc{pith2026260806799,
  author       = {Pith},
  title        = {Pith review of: Is Forward Prediction Enough? Physical State Grounding for JEPA World Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZU4NBCR2}},
  note         = {Machine review of arXiv:2608.06799}
}
read the original abstract

Learning structured and control-relevant latent representations remains a key challenge for world models. Recent JEPA-based world models learn action-conditioned predictive latent dynamics from observation sequences. However, their forward-prediction objectives do not explicitly enforce reliable identifiability of robot-centric physical state from individual latents or state changes from latent pairs, which can limit downstream planning and policy performance. We propose PSG-JEPA, a physically grounded JEPA world model that shapes its latent space with two complementary grounding objectives beyond forward prediction: grounding individual latents in robot proprioceptive state, and grounding latent pairs in multi-horizon joint-angle changes. Both objectives are applied only during training, leaving the inference architecture and computational cost unchanged. To comprehensively evaluate PSG-JEPA, we conduct experiments at three levels: (1) latent identifiability via probing, (2) goal-conditioned planning on frozen latents, and (3) policy learning in simulation and on a real robot. Experiments demonstrate that our PSG-JEPA consistently outperforms state-of-the-art latent world-model baselines at all three levels.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 5 canonical work pages

  1. [4]

    Ctrl-World: A controllable generative world model for robot manipulation.arXiv preprint arXiv:2510.10125,

    Yanjiang Guo, Lucy Xiaoyang Shi, Jianyu Chen, and Chelsea Finn. Ctrl-World: A controllable generative world model for robot manipulation.arXiv preprint arXiv:2510.10125,

    arXiv

  2. [5]

    World models.arXiv preprint arXiv:1803.10122, 2(3):440,

    David Ha and Jürgen Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2(3):440,

    arXiv

  3. [7]

    World model for robot learning: A comprehensive survey.arXiv preprint arXiv:2605.00080,

    Bohan Hou, Gen Li, Jindou Jia, Tuo An, Xinying Guo, Sicong Leng, Haoran Geng, Yanjie Ze, Tatsuya Harada, Philip Torr, et al. World model for robot learning: A comprehensive survey.arXiv preprint arXiv:2605.00080,

    arXiv

  4. [8]

    GAIA-1: A generative world model for autonomous driving.arXiv preprint arXiv:2309.17080,

    Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. GAIA-1: A generative world model for autonomous driving.arXiv preprint arXiv:2309.17080,

    arXiv

  5. [9]

    PAIWorld: A 3D-consistent world foundation model for robotic manipulation.arXiv preprint arXiv:2606.18375,

    Yuhang Huang, Xuan Lv, Junyan Xu, Zhiyuan Yu, Jiazhao Zhang, Ruizhen Hu, Wancheng Feng, Shilong Zou, Hewen Xiao, Ziqiao Zhou, et al. PAIWorld: A 3D-consistent world foundation model for robotic manipulation.arXiv preprint arXiv:2606.18375,

    arXiv

  6. [10]

    Robots pre-train robots: Manipulation-centric robotic representation from large-scale robot datasets

    Guangqi Jiang, Yifei Sun, Tao Huang, Huanyu Li, Yongyuan Liang, and Huazhe Xu. Robots pre-train robots: Manipulation-centric robotic representation from large-scale robot datasets. InInternational Conference on Learning Representations, volume 2025, pages 81885–81905,

    work page 2025

  7. [11]

    Contrastive representation regularization for vision-language-action models.arXiv preprint arXiv:2510.01711,

    Taeyoung Kim, Jimin Lee, Myungkyu Koo, Dongyoung Kim, Kyungmin Lee, Changyeon Kim, Younggyo Seo, and Jinwoo Shin. Contrastive representation regularization for vision-language-action models.arXiv preprint arXiv:2510.01711,

    arXiv

  8. [12]

    Predictive but not plannable: RC-aux for latent world models.arXiv preprint arXiv:2605.07278,

    Wenyuan Li, Guang Li, Keisuke Maeda, Takahiro Ogawa, and Miki Haseyama. Predictive but not plannable: RC-aux for latent world models.arXiv preprint arXiv:2605.07278,

    arXiv

Show all 29 references
  1. [13]

    Genie envisioner: A unified world foundation platform for robotic manipulation.arXiv preprint arXiv:2508.05635,

    Yue Liao, Pengfei Zhou, Siyuan Huang, Donglin Yang, Shengcong Chen, Yuxin Jiang, Yue Hu, Jingbin Cai, Si Liu, Jianlan Luo, et al. Genie envisioner: A unified world foundation platform for robotic manipulation.arXiv preprint arXiv:2508.05635,

    arXiv

  2. [14]

    Lda-1b: Scaling latent dynamics action model via universal embodied data ingestion

    Jiangran Lyu, Kai Liu, Xuheng Zhang, Haoran Liao, Yusen Feng, Wenxuan Zhu, Tingrui Shen, Jiayi Chen, Jiazhao Zhang, Yifei Dong, et al. Lda-1b: Scaling latent dynamics action model via universal embodied data ingestion. arXiv preprint arXiv:2602.12215,

    arXiv

  3. [15]

    LeWorldModel: Stable end-to-end joint-embedding predictive architecture from pixels.arXiv preprint arXiv:2603.19312,

    Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, and Randall Balestriero. LeWorldModel: Stable end-to-end joint-embedding predictive architecture from pixels.arXiv preprint arXiv:2603.19312,

    arXiv

  4. [16]

    V-JEPA 2.1: Unlocking dense features in video self-supervised learning.arXiv preprint arXiv:2603.14482,

    Lorenzo Mur-Labadia, Matthew Muckley, Amir Bar, Mido Assran, Koustuv Sinha, Mike Rabbat, Yann LeCun, Nicolas Ballas, and Adrien Bardes. V-JEPA 2.1: Unlocking dense features in video self-supervised learning.arXiv preprint arXiv:2603.14482,

    arXiv

  5. [17]

    Latent geometry beyond search: Amortizing planning in world models.arXiv preprint arXiv:2605.08732,

    Hoang Nguyen, Xiaohao Xu, and Xiaonan Huang. Latent geometry beyond search: Amortizing planning in world models.arXiv preprint arXiv:2605.08732,

    arXiv

  6. [18]

    LARY: A latent action representation yielding benchmark for generalizable vision-to-action alignment.arXiv preprint arXiv:2604.11689,

    Dujun Nie, Fengjiao Chen, Qi Lv, Jun Kuang, Xiaoyu Li, Xuezhi Cao, and Xunliang Cai. LARY: A latent action representation yielding benchmark for generalizable vision-to-action alignment.arXiv preprint arXiv:2604.11689,

    arXiv

  7. [19]

    Reconstruction or semantics? what makes a latent space useful for robotic world models, 2026.https://arxiv.org/abs/2605.06388

    Nilaksh, Saurav Jha, Artem Zholus, and Sarath Chandar. Reconstruction or semantics? what makes a latent space useful for robotic world models, 2026.https://arxiv.org/abs/2605.06388. Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre...

    2026 arXiv

  8. [20]

    OGBench: Benchmarking offline goal-conditioned RL

    Seohong Park, Kevin Frans, Benjamin Eysenbach, and Sergey Levine. OGBench: Benchmarking offline goal-conditioned RL. InInternational Conference on Learning Representations, volume 2025, pages 94937–94982,

    2025

  9. [21]

    Vlad Sobal, Wancong Zhang, Kyunghyun Cho, Randall Balestriero, Tim G. J. Rudner, and Yann LeCun. Stress-testing offline reward-free reinforcement learning: A case for planning with latent dynamics models. In7th Robot Learning Workshop: Towards Robots with Human-Level Abilities...

    2025

  10. [22]

    GigaWorld-0: World models as data engine to empower embodied AI.arXiv preprint arXiv:2511.19861,

    GigaWorld Team, Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jiagang Zhu, Kerui Li, Mengyuan Xu, et al. GigaWorld-0: World models as data engine to empower embodied AI.arXiv preprint arXiv:2511.19861,

  11. [23]

    Gigaworld-1: A roadmap to build world models for robot policy evaluation.arXiv preprint arXiv:2607.02642,

    GigaWorld Team, Angyuan Ma, Boyuan Wang, Bohan Li, Chaojun Ni, Guo Li, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, et al. Gigaworld-1: A roadmap to build world models for robot policy evaluation.arXiv preprint arXiv:2607.02642,

    arXiv

  12. [24]

    Beyond language modeling: An exploration of multimodal pretraining

    Shengbang Tong, David Fan, John Nguyen, Ellis Brown, Gaoyue Zhou, Shengyi Qian, Boyang Zheng, Théophane Vallaeys, Junlin Han, Rob Fergus, et al. Beyond language modeling: An exploration of multimodal pretraining. arXiv preprint arXiv:2603.03276,

  13. [25]

    Open-world hand-object interaction video generation based on structure and contact-aware representation

    Haodong Yan, Hang Yu, Zhide Zhong, Weilin Yuan, Xin Gong, Zehang Luo, Chengxi Heyu, Junfeng Li, Wenxuan Song, Shunbo Zhou, et al. Open-world hand-object interaction video generation based on structure and contact-aware representation. InProceedings of the IEEE/CVF Conference o...

    2024

  14. [26]

    UniDriveDreamer: A single-stage multimodal world model for autonomous driving.arXiv preprint arXiv:2602.02002,

    Guosheng Zhao, Yaozeng Wang, Xiaofeng Wang, Zheng Zhu, Tingdong Yu, Guan Huang, Yongchen Zai, Ji Jiao, Changliang Xue, Xiaole Wang, et al. UniDriveDreamer: A single-stage multimodal world model for autonomous driving.arXiv preprint arXiv:2602.02002,

  15. [27]

    FlowVLA: Visual chain of thought-based motion reasoning for vision-language-action models.arXiv preprint arXiv:2508.18269,

    Zhide Zhong, Haodong Yan, Junfeng Li, Xiangchen Liu, Xin Gong, Tianran Zhang, Wenxuan Song, Jiayi Chen, Xinhu Zheng, Hesheng Wang, et al. FlowVLA: Visual chain of thought-based motion reasoning for vision-language-action models.arXiv preprint arXiv:2508.18269,

  16. [28]

    DualCoT-VLA: Visual-linguistic chain of thought via parallel reasoning for vision-language-action models.arXiv preprint arXiv:2603.22280,

    Zhide Zhong, Junfeng Li, Junjie He, Haodong Yan, Xin Gong, Guanyi Zhao, Yingjie Cai, Jiantao Gao, Xu Yan, Bingbing Liu, Yingcong Chen, Liuqing Yang, and Haoang Li. DualCoT-VLA: Visual-linguistic chain of thought via parallel reasoning for vision-language-action models.arXiv pr...

  17. [29]

    DINO-WM: World models on pre-trained visual features enable zero-shot planning.arXiv preprint arXiv:2411.04983,

    Gaoyue Zhou, Hengkai Pan, Yann LeCun, and Lerrel Pinto. DINO-WM: World models on pre-trained visual features enable zero-shot planning.arXiv preprint arXiv:2411.04983,

    arXiv

  18. [2019]

    Dream to control: Learning behaviors by latent imagination

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. InInternational Conference on Learning Representations, 2020.https://openreview.net/forum? id=S1lOTC4tDS. Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, ...

    2020 arXiv

  19. [2023]

    V-JEPA 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985, 2025.https://arxiv.org/abs/2506.09985

    Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, Sergio Arnaud, Abha Gejji, Ada Martin, Francois Robert Hogan, Daniel Dugas, Piotr Bojanowski, Vasil Khalidov, Patr...

    2025 arXiv

  20. [2025]

    Why ai systems don’t learn and what to do about it: Lessons on autonomous learning from cognitive science.arXiv preprint arXiv:2603.15381,

    Emmanuel Dupoux, Yann LeCun, and Jitendra Malik. Why ai systems don’t learn and what to do about it: Lessons on autonomous learning from cognitive science.arXiv preprint arXiv:2603.15381,

  21. [2026]

    Mobile ALOHA: Learning bimanual mobile manipulation with low-cost whole-body teleoperation.arXiv preprint arXiv:2401.02117,

    Zipeng Fu, Tony Z Zhao, and Chelsea Finn. Mobile ALOHA: Learning bimanual mobile manipulation with low-cost whole-body teleoperation.arXiv preprint arXiv:2401.02117,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.