Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

RoboEnvision: A Long-Horizon Video Generation Model for Multi-Task Robot Manipulation

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that long-horizon robot manipulation videos can be generated by decomposing a high-level instruction into per-task keyframes and interpolating between them, bypassing autoregressive chaining and its accumulated errors.

desk verdict A sensible hierarchical video-generation pipeline for long-horizon manipulation, with real architectural ideas, but the headline success number is not yet cleanly attributed because the VLM planning stage is unquantified and the evaluation is thin on error bars and external validation. read the letter →

arxiv 2506.22007 v1 pith:YO662YGW submitted 2025-06-27 cs.CV

Liudi Yang , Yang Bai , George Eskandar , Fengyi Shen , Mohammad Altillawi , Dong Chen , Soumajit Majumder , Ziyuan Liu
show 2 more authors
Gitta Kutyniok Abhinav Valada
This is my paper
classification cs.CV
keywords long-horizonvideogenerationrobotmanipulationdiffusionmodelkeyframevision-languageplanningvideo-conditionedpolicysemanticspreservingattentionMuJoCobenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RoboEnvision tries to establish that long-horizon robotic manipulation videos can be generated by breaking a high-level instruction into atomic sub-instructions, generating one keyframe per sub-instruction, and then interpolating between keyframes, without chaining short video segments autoregressively. The paper argues that this hierarchy, combined with a consistency-enforcing attention module, removes the accumulated drift that limits prior video-planning models. To make the videos actionable, a lightweight transformer policy regresses robot joint states from the generated frames. On the LanguageTable and LHMM benchmarks, the paper reports the best video-quality metrics, and on LHMM long-horizon execution it reports 67.4% success versus 23.5% for UniPi and 34.1% for RDT1B. If correct, this would make generated video a practical intermediate representation for multi-step robot planning and data augmentation.

What carries the argument

The load-bearing mechanism is a two-stage hierarchical diffusion pipeline. Stage one is a keyframe diffusion model, built on the OpenSora DiT architecture, that receives a chain of atomic instructions and outputs one keyframe per instruction; it uses a diagonal-block mask in cross-attention so each keyframe is conditioned only on its own instruction, and it replaces temporal attention with 3D full attention to tolerate large motions between keyframes. A semantics-preserving attention module adds a cross-attention branch that injects the VAE features of the initial observation into each spatial attention layer, which keeps small objects from changing shape or disappearing. Stage two is a filling diffusion model that interpolates between consecutive keyframes to produce the full long video. Finally, a lightweight spatio-temporal transformer with a ResNet decoder regresses joint configurations from the keyframes plus selected interpolated frames, enabling open-loop execution.

What would settle it

On LHMM, run the full pipeline but replace the keyframe diffusion outputs with ground-truth keyframes from the simulator, keeping the same instructions, filling model, and policy: if success does not rise clearly above the reported 67.4%, the claim that the generated video's fidelity drives the policy result is falsified; conversely, if a random or deliberately wrong VLM decomposition collapses success, the planning step is confirmed as load-bearing.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that the bottleneck for long-horizon video planning is not frame count but consistency across large motions, and that consistency can be restored by explicitly aligning each keyframe to one sub-instruction and re-injecting the first image's VAE features into spatial attention. The proposed RoboEnvision pipeline predicts a sparse set of keyframes, each representing the end state of an atomic instruction such as 'pick up the lemon' or 'place it in box 1,' and a second filling diffusion model interpolates between consecutive keyframes. A masked keyframe-instruction cross-attention ensures each keyframe attends only to its own instruction, while a semantics-preserving attention module injects the first frame's VAE features to preserve object shape and count. The generated video is then read by a lightweight spatio-temporal transformer policy that regresses joint angles and gripper state, executed open-loop in MuJoCo. The paper claims this design outperforms prior autoregressive and hierarchical baselines on video-quality metrics and, by a large margin, on long-horizon task success.

Load-bearing premise

The pipeline assumes the vision-language model decomposes the high-level instruction into atomic sub-instructions that are feasible and correctly ordered; if that decomposition is wrong, the keyframes, interpolated video, and joint trajectories all inherit the error, so total performance is bounded by the planner's accuracy.

Editorial extensions

If this is right

  • Autoregressive chaining is no longer required for long-horizon video planning: one keyframe pass plus parallel interpolation produces the full video.
  • Generated long-horizon videos can serve directly as robot policies through a lightweight joint-regression model, achieving 67.4% success on LHMM without VLA-scale pretraining.
  • Reordering the atomic instruction chain reorders the execution, so the same dataset can be expanded into new task orders with matching joint states.
  • Open-loop execution success on long-horizon tasks becomes a measurable downstream target alongside video metrics like LPIPS and FVD.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the practical ceiling is set by the VLM planner: if it produces an infeasible or misordered instruction chain, the keyframe model, filling model, and policy inherit the error, so a useful stress test is to swap GPT-4o1 for random or rule-based planners and compare success rates.
  • Beyond the paper, the simulated, open-loop evaluation does not establish physical-robot transfer; the policy is trained on simulator ground-truth joint states, so deployment would require the video model or policy to bridge the sim-to-real gap.
  • Beyond the paper, the keyframe-plus-interpolation recipe is not obviously robot-specific and could be tried on other long-form generation tasks where autoregressive drift is the dominant failure mode, such as multi-step human activity videos.
  • Beyond the paper, substituting ground-truth simulator keyframes into the filling and policy stages would isolate how much of the reported success comes from video generation fidelity rather than from the planner and policy design.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes RoboEnvision, a three-stage pipeline for long-horizon robot manipulation video generation: (i) a VLM decomposes a high-level instruction into atomic sub-instructions; (ii) a keyframe diffusion model generates one frame per sub-instruction using a masked keyframe-instruction cross-attention and a semantics-preserving attention module that injects first-frame VAE features; (iii) a filling diffusion model interpolates between keyframes. A lightweight spatio-temporal transformer policy regresses joint states from the generated frames and is evaluated open-loop in MuJoCo. The paper reports state-of-the-art video-quality metrics on LanguageTable and a new LHMM dataset, and 67.4% success on 45 LHMM tasks versus 23.5% for UniPi and 34.1% for RDT1b.

Significance. If the results hold, the pipeline is a useful step: it avoids autoregressive error accumulation via coarse-to-fine keyframe generation, and the ablations in Table III support the contribution of 3D attention and semantics-preserving attention. Strengths: the policy model is trained on independent simulator ground truth rather than on the video model's outputs, which mitigates circularity; the open-loop MuJoCo evaluation is a concrete downstream test; and the keyframe-instruction masking and VAE-feature reinjection are clean, implementable ideas. The significance is tempered by the unquantified VLM stage and by the fact that the main policy comparison is on a self-created benchmark with no error bars, so the headline 'state-of-the-art' is not yet fully supported.

major comments (3)
  1. [Section III-B, IV-D] The first pipeline stage—VLM-based decomposition of the high-level instruction into sub-instructions—is never quantitatively evaluated. The paper only shows qualitative examples in Fig. 5, and Eq. (3) makes the filling diffusion conditioned on the sub-instruction l_i, so a wrong or infeasible decomposition propagates into the keyframes, interpolated frames, and joint predictions. The 67.4% success rate in Table II is therefore an end-to-end number that bundles GPT-4o1's planning with the video and policy stages. Please report decomposition accuracy or run a control experiment in which the same keyframe/filling/policy stages use ground-truth or human-verified decompositions, and provide a per-task success breakdown; without this, the claim that the proposed video and policy contributions outperform prior policy models is not isolated from the VLM.
  2. [Section IV-C, Table II] The policy comparison is not yet convincing. RDT1b is finetuned for only 11k steps on 200 episodes and UniPi is a from-text reproduction, so the baseline strength is uncertain; no standard deviation or number of trials is reported for any success rate; and it is not stated whether all methods receive the same generated videos and the same VLM decomposition. Please evaluate all policies on identical generated videos and a common instruction set, run multiple seeds, report per-task results, and include an upper-bound policy trained and evaluated on ground-truth videos to separate video-generation errors from policy-regression errors.
  3. [Section IV-A, Table I] The 'state-of-the-art on two benchmarks' claim rests on metrics computed on 124 and 100 generated videos, with no error bars, and one of the two benchmarks (LHMM) is introduced in this paper. The absence of variance and statistical testing makes the four-out-of-five metric improvements fragile, especially where margins are small (e.g., LPIPS 0.1324 vs 0.1445 on LanguageTable). Please provide confidence intervals or multiple-seed results, and clarify whether the LHMM benchmark and its evaluation protocol will be released; the same issue applies to the ablation results in Table III.
minor comments (6)
  1. [Equation (1)] The expression 'l_HL = K M i=1 l_i' uses an undefined symbol; this should be a concatenation or product with a clear index range, and Eq. (5) has a similar issue with 'K M i=1 tau_i'.
  2. [Section II] 'astronautical-scale pretraining' should be 'astronomical-scale pretraining'.
  3. [Fig. 2 caption] 'short-horzion' should be 'short-horizon'.
  4. [Table III] The table does not state which dataset and how many videos are used; please specify.
  5. [Section IV-A] The statement that dataset sizes are 50k and 90k does not specify whether these are clips, frames, or samples; please clarify.
  6. [Table I] The baseline label 'A VDC' should be made consistent with the notation used in the text and references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the policy and video stages are trained on ground-truth simulator data and evaluated against external baselines; the author-introduced LHMM benchmark limits external grounding but does not make the derivation circular.

full rationale

The paper's central claims are empirical measurements rather than derivations from fitted constants. The policy model is explicitly trained on ground-truth simulator data independently of the video generation model (Sec. III-F: 'The model is trained independently on the groundtruth data provided from the simulator, independently from the video model'), so the 67.4% success rate on LHMM is an outcome of executing predicted joint states in MuJoCo, not a quantity forced by construction. The video quality comparisons in Table I are against OpenSora variants and AVDC, which are external baselines, and the keyframe/filling pipeline is a generative architecture whose consistency is measured with standard metrics. The VLM decomposition stage is evaluated only qualitatively, and an incorrect decomposition could propagate errors downstream, but this is a missing-evidence and robustness concern, not a circularity: the paper does not define the video or policy outputs in terms of the VLM's decomposed instructions, and the policy model is trained independently of that stage. The main self-referential element is that LHMM is both introduced by the authors and used as the evaluation benchmark for the headline success rate; this limits external grounding and comparability, but it is not a logical reduction of the result to its own inputs. No fitted parameter is renamed as a prediction, no uniqueness claim is imported from the authors' prior work, and no load-bearing self-citation appears in the derivation chain. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper is an empirical systems contribution; the central claims rest mainly on domain assumptions rather than derived physical laws. Key hand-chosen settings (prompt template, optical flow threshold, segment frame counts) are not reported with sensitivity analysis. No new physical entities are introduced; the LHMM dataset is a new benchmark artifact, not released.

free parameters (3)
  • VLM prompt template
    The instruction decomposition from GPT-4o1 is steered by a hand-designed in-context template (Sec. IV-A). No sensitivity analysis is reported, and the template directly determines the keyframe sequence.
  • Optical flow consistency threshold
    LanguageTable long-horizon training videos are created by concatenating clips 'using optical flow consistency checks' (Sec. IV-A). The threshold is not specified; if too permissive, stitched videos contain physically impossible transitions.
  • Frames per filling segment F_i
    The filling diffusion generates F_i frames per keyframe pair (Sec. III-E), affecting both the total video length and the policy model's input spacing. No ablation is provided for this choice.
assumptions (5)
  • standard math Diffusion training objective and noise schedule follow standard DDPM/OpenSora practice (Eq. 4-5).
    The method relies on the established diffusion formulation without modification.
  • domain assumption LanguageTable short-horizon clips can be concatenated into valid long-horizon training videos via optical flow consistency checks.
    This assumption underlies all LanguageTable training data (Sec. IV-A); if false, the keyframe and filling models learn unrealistic transitions.
  • domain assumption GPT-4o1's decomposition of high-level instructions produces correct, executable sub-tasks.
    The entire pipeline is conditioned on the VLM's instruction chain (Sec. III-B, Sec. IV-D). Only qualitative evidence is provided.
  • ad hoc to paper Injecting first-frame VAE features into every spatial attention block preserves object shape without degrading motion quality.
    This is the key design choice in Sec. III-D; the paper validates it only through ablations on its own benchmarks.
  • domain assumption MuJoCo simulator dynamics are an adequate testbed for the long-horizon success-rate claim.
    All policy evaluations happen in simulation (Sec. IV-C); real-world transfer is not tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RoboEnvision: A Long-Horizon Video Generation Model for Multi-Task Robot Manipulation." pith.science (2026). https://pith.science/paper/YO662YGW

@misc{pith2026250622007,
  author       = {Pith},
  title        = {Pith review of: RoboEnvision: A Long-Horizon Video Generation Model for Multi-Task Robot Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YO662YGW}},
  note         = {Machine review of arXiv:2506.22007}
}
read the original abstract

We address the problem of generating long-horizon videos for robotic manipulation tasks. Text-to-video diffusion models have made significant progress in photorealism, language understanding, and motion generation but struggle with long-horizon robotic tasks. Recent works use video diffusion models for high-quality simulation data and predictive rollouts in robot planning. However, these works predict short sequences of the robot achieving one task and employ an autoregressive paradigm to extend to the long horizon, leading to error accumulations in the generated video and in the execution. To overcome these limitations, we propose a novel pipeline that bypasses the need for autoregressive generation. We achieve this through a threefold contribution: 1) we first decompose the high-level goals into smaller atomic tasks and generate keyframes aligned with these instructions. A second diffusion model then interpolates between each of the two generated frames, achieving the long-horizon video. 2) We propose a semantics preserving attention module to maintain consistency between the keyframes. 3) We design a lightweight policy model to regress the robot joint states from generated videos. Our approach achieves state-of-the-art results on two benchmarks in video quality and consistency while outperforming previous policy models on long-horizon tasks.

Figures

Figures reproduced from arXiv: 2506.22007 by the authors.

Figure 1
Figure 1. Top: Previous Works [3], [8] predict short-horizon videos and estimate robot actions from them. Long horizon tasks are executed by cascading this approach sequentially along the time axis. Bottom: Our RoboEnvision model, breaks down a high-level instruction into small atomic instructions with a VLM, generates a frame aligned with each one, and interpolates between them. A policy model estimates the robot joints base… view at source ↗
Figure 2
Figure 2. RoboEnvision generates keyframes aligned with short-horzion instructions (Stage 1) and interpolates between them (Stage 2). We show: (A) the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results comparing our method with baselines on the LanguageTable and LHMM datasets. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of long-horizon video generation based on instructions [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Qualitative results of the ablation study on the enhanced architec [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Qualitative results of long-horizon planning using GPT4-o1. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. QDepth-VLA: Quantized Depth Prediction as Auxiliary Supervision for Vision-Language-Action Models

    cs.CV 2025-10 conditional novelty 5.0 of 10

    Adding an auxiliary quantized-depth-token prediction task to a VLA policy improves manipulation success rates on LIBERO, Simpler, and real-robot pick-and-place tasks versus the open-pi-zero baseline.

Reference graph

Works this paper leans on

33 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    Video language planning,

    Y . Du, S. Yang, P. Florence, F. Xia, A. Wahid, brian ichter, P. Sermanet, T. Yu, P. Abbeel, J. B. Tenenbaum, L. P. Kaelbling, A. Zeng, and J. Tompson, “Video language planning,” in The Twelfth International Conference on Learning Representations , 2024

    work page 2024

  2. [2]

    This&that: Language-gesture controlled video generation for robot planning,

    B. Wang, N. Sridhar, C. Feng, M. Van der Merwe, A. Fish- man, N. Fazeli, and J. J. Park, “This&that: Language-gesture controlled video generation for robot planning,” arXiv preprint arXiv:2407.05530, 2024

    arXiv 2024

  3. [3]

    Learning universal policies via text- guided video generation,

    Y . Du, S. Yang, B. Dai, H. Dai, O. Nachum, J. B. Tenenbaum, D. Schuurmans, and P. Abbeel, “Learning universal policies via text- guided video generation,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023

    work page 2023

  4. [4]

    Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation,

    C.-L. Cheang, G. Chen, Y . Jing, T. Kong, H. Li, Y . Li, Y . Liu, H. Wu, J. Xu, Y . Yang, H. Zhang, and M. Zhu, “Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation,” 10 2024

    work page 2024

  5. [5]

    Learning interactive real-world simulators,

    S. Yang, Y . Du, S. K. S. Ghasemipour, J. Tompson, L. P. Kael- bling, D. Schuurmans, and P. Abbeel, “Learning interactive real-world simulators,” in The Twelfth International Conference on Learning Representations, 2024

    work page 2024

  6. [6]

    Open-sora: Democratizing efficient video production for all,

    Z. Zheng, X. Peng, T. Yang, C. Shen, S. Li, H. Liu, Y . Zhou, T. Li, and Y . You, “Open-sora: Democratizing efficient video production for all,” 2024

    work page 2024

  7. [7]

    Cogvideox: Text-to- video diffusion models with an expert transformer,

    Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y . Yang, W. Hong, X. Zhang, G. Feng et al. , “Cogvideox: Text-to- video diffusion models with an expert transformer,” arXiv preprint arXiv:2408.06072, 2024

    arXiv 2024

  8. [8]

    Learning to act from actionless videos through dense correspondences,

    P.-C. Ko, J. Mao, Y . Du, S.-H. Sun, and J. B. Tenenbaum, “Learning to act from actionless videos through dense correspondences,” arXiv preprint arXiv:2310.08576, 2023

    arXiv 2023

Show all 33 references
  1. [9]

    Streamingt2v: Consis- tent, dynamic, and extendable long video generation from text,

    R. Henschel, L. Khachatryan, D. Hayrapetyan, H. Poghosyan, V . Tade- vosyan, Z. Wang, S. Navasardyan, and H. Shi, “Streamingt2v: Consis- tent, dynamic, and extendable long video generation from text,” 2024

    2024

  2. [10]

    Gen-l- video: Multi-text to long video generation via temporal co-denoising,

    F.-Y . Wang, W. Chen, G. Song, H.-J. Ye, Y . Liu, and H. Li, “Gen-l- video: Multi-text to long video generation via temporal co-denoising,” 2023

    2023

  3. [11]

    VDT: General-purpose video diffusion transformers via mask modeling,

    H. Lu, G. Yang, N. Fei, Y . Huo, Z. Lu, P. Luo, and M. Ding, “VDT: General-purpose video diffusion transformers via mask modeling,” in The Twelfth International Conference on Learning Representations , 2024

    2024

  4. [12]

    pi 0: A vision- language-action flow model for general robot control,

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter et al. , “ pi 0: A vision- language-action flow model for general robot control,” arXiv preprint arXiv:2410.24164, 2024

    2024 arXiv

  5. [13]

    RDT-1b: a diffusion foundation model for bimanual ma- nipulation,

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “RDT-1b: a diffusion foundation model for bimanual ma- nipulation,” in The Thirteenth International Conference on Learning Representations, 2025

    2025

  6. [14]

    Diffusion policy: Visuomotor policy learning via ac- tion diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,” The International Journal of Robotics Research , p. 02783649241273668, 2023

    2023

  7. [15]

    Mujoco: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , 2012, pp. 5026–5033

    2012

  8. [16]

    Robo- dreamer: Learning compositional world models for robot imagination,

    S. Zhou, Y . Du, J. Chen, Y . Li, D.-Y . Yeung, and C. Gan, “Robo- dreamer: Learning compositional world models for robot imagination,” arXiv preprint arXiv:2404.12377 , 2024

    2024 arXiv

  9. [17]

    Genie: Generative interactive environments,

    J. Bruce, M. D. Dennis, A. Edwards, J. Parker-Holder, Y . Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps et al. , “Genie: Generative interactive environments,” in Forty-first Interna- tional Conference on Machine Learning , 2024

    2024

  10. [18]

    Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation,

    H. Bharadhwaj, D. Dwibedi, A. Gupta, S. Tulsiani, C. Doersch, T. Xiao, D. Shah, F. Xia, D. Sadigh, and S. Kirmani, “Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation,” arXiv preprint arXiv:2409.16283 , 2024

    2024 arXiv

  11. [19]

    Irasim: Learning interactive real-robot action simulators,

    F. Zhu, H. Wu, S. Guo, Y . Liu, C. Cheang, and T. Kong, “Irasim: Learning interactive real-robot action simulators,” arXiv preprint arXiv:2406.14540, 2024

    2024 arXiv

  12. [20]

    Any-point trajectory modeling for policy learning,

    C. Wen, X. Lin, J. So, K. Chen, Q. Dou, Y . Gao, and P. Abbeel, “Any-point trajectory modeling for policy learning,” 2023

    2023

  13. [21]

    Track2act: Predicting point tracks from internet videos enables diverse zero-shot robot manipulation,

    H. Bharadhwaj, R. Mottaghi, A. Gupta, and S. Tulsiani, “Track2act: Predicting point tracks from internet videos enables diverse zero-shot robot manipulation,” 2024

    2024

  14. [22]

    Freenoise: Tuning-free longer video diffusion via noise rescheduling,

    H. Qiu, M. Xia, Y . Zhang, Y . He, X. Wang, Y . Shan, and Z. Liu, “Freenoise: Tuning-free longer video diffusion via noise rescheduling,” in The Twelfth International Conference on Learning Representations, 2024

    2024

  15. [23]

    Freelong: Training-free long video generation with spectralblend temporal attention,

    Y . Lu, Y . Liang, L. Zhu, and Y . Yang, “Freelong: Training-free long video generation with spectralblend temporal attention,” in The Thirty- eighth Annual Conference on Neural Information Processing Systems , 2024

    2024

  16. [24]

    Nuwa-xl: Diffusion over diffusion for extremely long video generation,

    S. Yin, C. Wu, H. Yang, J. Wang, X. Wang, M. Ni, Z. Yang, L. Li, S. Liu, F. Yang, J. Fu, G. Ming, L. Wang, Z. Liu, H. Li, and N. Duan, “Nuwa-xl: Diffusion over diffusion for extremely long video generation,” 2023

    2023

  17. [25]

    Learning to reason with llms,

    OpenAI, “Learning to reason with llms,” 2024, ac- cessed: 2025-02-27. [Online]. Available: https://openai.com/index/ learning-to-reason-with-llms/

    2024

  18. [26]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    DeepSeek-AI, “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” 2025

    2025

  19. [27]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,

    H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang, “Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,” 2023

    2023

  20. [28]

    Interactive language: Talking to robots in real time,

    C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence, “Interactive language: Talking to robots in real time,” IEEE Robotics and Automation Letters , 2023

    2023

  21. [29]

    Robovqa: Multimodal long- horizon reasoning for robotics,

    P. Sermanet, T. Ding, J. Zhao, F. Xia, D. Dwibedi, K. Gopalakrishnan, C. Chan, G. Dulac-Arnold, S. Maddineni, N. J. Joshi, P. Florence, W. Han, R. Baruch, Y . Lu, S. Mirchandani, P. Xu, P. Sanketi, K. Haus- man, I. Shafran, B. Ichter, and Y . Cao, “Robovqa: Multimodal long- ho...

    2023 arXiv

  22. [30]

    Vlabench: A large-scale benchmark for language- conditioned robotics manipulation with long-horizon reasoning tasks,

    S. Zhang, Z. Xu, P. Liu, X. Yu, Y . Li, Q. Gao, Z. Fei, Z. Yin, Z. Wu, Y .-G. Jianget al., “Vlabench: A large-scale benchmark for language- conditioned robotics manipulation with long-horizon reasoning tasks,” arXiv preprint arXiv:2412.18194 , 2024

    2024 arXiv

  23. [31]

    LHManip: A dataset for long-horizon language-grounded manipulation tasks in clut- tered tabletop environments,

    F. Ceola, L. Natale, N. Suenderhauf, and K. Rana, “LHManip: A dataset for long-horizon language-grounded manipulation tasks in clut- tered tabletop environments,” inRSS 2024 Workshop: Data Generation for Robotics, 2024

    2024

  24. [32]

    Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks,

    O. Mees, L. Hermann, E. Rosete-Beas, and W. Burgard, “Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks,” IEEE Robotics and Automation Letters , vol. 7, no. 3, pp. 7327–7334, 2022

    2022

  25. [33]

    Lohoravens: A long-horizon language- conditioned benchmark for robotic tabletop manipulation,

    S. Zhang, P. Wicke, L. K. S ¸enel, L. Figueredo, A. Naceri, S. Haddadin, B. Plank, and H. Sch ¨utze, “Lohoravens: A long-horizon language- conditioned benchmark for robotic tabletop manipulation,” arXiv preprint arXiv:2310.12020, 2023

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.