Pith. sign in

REVIEW 4 major objections 5 minor 55 references

TeViR: Text-to-Video Reward with Diffusion Models for Efficient Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A text-to-video diffusion model can supply dense rewards for robotic RL.

desk verdict A genuinely new dense-reward mechanism from text-to-video generation with real sim evidence, but the real-world 'no environmental reward' headline is not tested. read the letter →

arxiv 2505.19769 v2 pith:3C24JPFC submitted 2025-05-26 cs.RO cs.AI

Yuhui Chen , Haoran Li , Zhennan Jiang , Haowei Wen , Dongbin Zhao This is my paper · ORCID
classification cs.ROcs.AI
keywords rewardengineeringtext-to-videodiffusiondensereinforcementlearningroboticmanipulationsampleefficiencymulti-viewobservationsvideoprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to show that reward engineering for robotic RL can be replaced by a text-to-video diffusion model: condition the model on the current camera view and a language instruction, generate a short imagined video of successful task execution, and compare each new observation with those imagined frames to produce a dense reward. The payoff claimed is that an RL agent can learn complex manipulation skills—pushing, pulling, grasping, and opening and closing doors, drawers, and windows—without any ground-truth environment reward, and with better sample efficiency than sparse-reward methods or earlier video-based reward schemes. The paper reports results on 11 Meta-World simulated tasks and 2 real-world tasks, where the dense reward yields higher success rates within the same training budget, and it also shows some tolerance to corrupted video generation. If right, this would remove a major bottleneck in applying RL to real robots: hand-crafted or environment-provided rewards would no longer be required.

What carries the argument

The load-bearing object is the dense reward computed from a generated video. The video model encodes three views—left, top, and close—with a VQ-GAN encoder, stacks the latents into one frame, and uses a Video U-Net conditioned on the initial frame and a CLIP-encoded language instruction to synthesize $H=8$ future frames. The reward then has three parts. The distance part $\sigma(z_t,\hat z_{h^*})$ is a view-weighted cosine similarity between the current latent observation and the generated frame it most resembles; the progress part uses a 'reached image' counter $M_t$ that increments when the current observation is similar enough, above a threshold $\theta$, to the next unreached generated frame, which prevents the agent from skipping intermediate steps; the exploration part is random-network distillation. The paper's argument is that comparing against a whole predicted trajectory, rather than a single next frame, is what makes the reward dense and long-horizon.

What would settle it

Freeze a trained video model and evaluate the TeViR reward on scripted rollouts where the object position, gripper pose, or task stage is systematically perturbed and ground-truth progress is known: if the reward does not strictly order those rollouts by true progress—for instance, if a rollout that never touches the object scores higher than one that completes the task—the central claim fails. A cheaper check is to feed the model an initial frame that makes the task impossible, such as an object absent or a drawer already open, and observe whether the reward still increases as the agent acts.

Watch

Extended reading notes

Core claim

TeViR's central claim is that a single text-to-video diffusion model can act as a dense reward source for visual RL. At rollout time, the model receives the first multi-view image $z_0$ and a natural-language task description, and generates an $H=8$-frame sequence $\{\hat z_0,\dots,\hat z_{H-1}\}$ representing an imagined expert trajectory. For each observed state $z_t$, the reward is $r_t^{\mathrm{TeViR}} = r^{\mathrm{dist}}_t + r^{\mathrm{prog}}_t + r^{\mathrm{expl}}_t$: a distance reward equal to the cosine similarity between $z_t$ and the closest generated frame up to the current 'reached' index; a progress reward that increases when the observation matches generated frames further along the sequence, with a binary completion bonus when the final frame is matched; and a random-network-distillation exploration bonus. When an environment sparse reward exists, the binary completion term can be replaced by that ground-truth signal, giving the variant the paper calls TeViR+. The experiments across 11 Meta-World tasks and two real-world tasks are the evidence offered that this reward works without environment feedback and improves credit assignment relative to sparse reward alone.

Load-bearing premise

The premise that must hold for TeViR to work is that the generated video frames are a faithful, temporally ordered blueprint of successful task execution, and that cosine similarity in the compressed latent space is a trustworthy proxy for how much task progress the agent has made; the per-task similarity thresholds and per-view weights are hand-set, so those choices are part of the same assumption.

Editorial extensions

If this is right

  • If the central claim is right, visual RL for manipulation can be trained from a language instruction plus a short expert-video collection, without any hand-coded or environment-supplied reward.
  • Because TeViR+ still uses the environment sparse reward when available, it offers a direct way to accelerate existing sparse-reward pipelines rather than only replacing them.
  • The robustness experiments suggest that imperfect or noisy generated videos are tolerable up to a point, so the reward does not require a perfect world model.
  • The same distance-plus-progress reward formula could be applied to new tasks by retraining or fine-tuning the text-to-video model on roughly 30 expert videos per task.
  • Multi-view observation is part of the recipe: combining left, top, and close views mitigates occlusion and improves the stability of the similarity-based reward.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: reward quality is bounded by the video model's semantic fidelity, so a natural extension is to fine-tune the video model on object-centric or task-specific representations and measure whether reward-to-progress correlation improves.
  • My inference: the distance-and-progress decomposition behaves like a soft subgoal curriculum, which could be used to detect which stage of a task the agent is stuck at and to trigger human intervention or automated curriculum changes.
  • My inference: because the reward is defined in latent space, it may transfer across robot embodiments if the video model can generate the new embodiment's visuals; testing cross-embodiment transfer would be a low-cost next experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TeViR, a dense reward function for reinforcement learning that uses a text-to-video diffusion model to synthesize a future image sequence from the current observation and a language description. The reward is computed as a weighted sum of (i) a distance reward based on cosine similarity between the current latent observation and the best-matching generated frame, (ii) a progress reward that tracks how far along the generated sequence the agent has reached, and (iii) a Random Network Distillation exploration bonus. The method is evaluated on Meta-World and two real-world robotic manipulation tasks, with comparisons against sparse reward, RoboCLIP, VIPER, Diffusion Reward, and UniPi. The authors claim improved sample efficiency and success rates, and highlight that TeViR works 'without ground truth environmental rewards.'

Significance. If validated, TeViR would be a useful contribution to reward engineering for robotic manipulation: it replaces hand-designed dense rewards with a video-diffusion-based signal, and the idea of using long-horizon generated trajectories for reward shaping is timely and potentially impactful. The paper includes extensive experiments (8 Meta-World tasks in the main figures, plus real-world experiments), ablations of each reward component, a multi-view analysis, and a robustness study against noisy or erroneous generated videos. These are meaningful strengths. However, the central 'no ground-truth reward' claim is only directly tested in simulation, the real-world evaluation actually uses TeViR+ (which includes the environmental sparse reward), and the comparison does not control for the RND exploration bonus that TeViR includes but the baselines lack. The per-task thresholds and view weights in Table I also weaken the generality claim. Overall, the core idea is promising but the evidence as presented does not fully support the headline claims.

major comments (4)
  1. [Abstract and Section V-E] The abstract and Contribution 3 claim that TeViR achieves a 29.0% average success-rate improvement on real-world tasks 'without the environmental rewards.' However, Section V-E evaluates only TeViR+ (Equation 6), which explicitly replaces the indicator term with r_spar, the ground-truth sparse reward generated by the ResNet classifier. No real-world experiment runs TeViR (Equation 5) alone. The real-world evidence therefore does not support the no-environmental-reward claim; that claim rests entirely on the Meta-World experiments in Section V-C. This is a direct mismatch between the evidence and the headline result, and it should be fixed either by running TeViR without the sparse reward in the real world or by revising the claim.
  2. [Section V-B, V-C, and Figures 5, 7] The paper states in Contribution 3 that experiments span '11 Meta-World tasks,' and Table I lists 11 Meta-World tasks. Yet the main learning curves in Figures 5 and 7 show only 8 tasks. The text in Sections V-B and V-C also says 'we select 8 tasks.' If the remaining 3 tasks were evaluated, their results need to be reported (e.g., in a table or appendix); if not, the claim of '11 Meta-World tasks' is unsupported. The discrepancy between the stated scope and the presented evidence is load-bearing because the paper's central claim is about broad effectiveness across diverse tasks.
  3. [Section IV-C, Equation (5), and Section V-C] The TeViR reward in Equation (5) includes the exploration reward r_expl (Random Network Distillation), but the baselines RoboCLIP, VIPER, and Diffusion Reward- are evaluated without any comparable exploration bonus. The observed sample-efficiency improvement in Figure 7 could therefore be partly due to RND rather than the text-to-video dense reward. The ablation in Figure 13 shows that removing r_expl hurts performance, but this does not address the confound. Please provide a comparison where the baselines are augmented with an equivalent intrinsic-reward term, or otherwise isolate the contribution of the video-based reward.
  4. [Table I and Section IV-C] The progress and distance rewards depend on per-task hyperparameters: the similarity threshold θ and the view weights {w_left, w_top, w_close} are listed separately for each of the 11 tasks in Table I. The text says these are 'adjusted according to specific environment and task requirements.' This undermines the Conclusion's statement that TeViR works 'without the need for task-specific reward engineering.' Please either report a sensitivity analysis showing that performance is robust to these hyperparameters, or describe a principled procedure for setting them, and temper the generality claim accordingly.
minor comments (5)
  1. [Abstract] The abstract in the provided text says '11 complex robotic tasks,' while the full-text abstract in Section I says '13 simulation and real-world robotic tasks.' Please align the numbers across the paper.
  2. [Section IV-C, Equation (2)] The summation notation in Equation (2) is garbled: 'PP i=0 w_viewi σ(z_viewi_t, bz_viewi_h)' should be written with a clear index and bounds, e.g., sum_{i=1}^{P} w_i σ(z_i_t, bz_i_h), and the subscript 'p' in the following sentence should be 'i.'
  3. [Algorithm 1 and Figure 3] Algorithm 1 initializes M_0 = 0, but the definition of h* in Equation (4) uses an arg max over {0, ..., M_t - 1}, which is empty at t = 0. Also, the progress-update condition refers to σ(z_t, bz_{M_t - 1}), which would reference bz_{-1} when M_t = 0. Please correct the off-by-one indexing so that the algorithm is well-defined.
  4. [Table I and Section V-E] The real-world task names are inconsistent: Table I lists 'Pick Carrot' and 'Pull Compartment,' while Figure 6 and the Section V-E text describe 'Pick Carrot' and 'Open Drawer.' Please harmonize the task names.
  5. [Section V-E] The real-world experiments use HIL-SERL with human intervention. The paper says intervention is 'consistent' across settings, but no quantitative measure of the number or timing of human interventions is provided. Please report this detail to ensure the comparison isolates the reward signal.

Circularity Check

0 steps flagged · score 0.0 of 10

TeViR is an empirical reward-shaping method; no load-bearing step reduces the reward derivation to its own inputs or to a self-citation chain.

full rationale

TeViR's reward is constructed in Equations (2)-(6) as cosine similarity in a VQ-GAN latent space between the current observation and frames generated by a text-to-video model, plus a progress counter and an RND exploration term. The generator is trained per task on expert demonstrations (Section V-A), not on RL returns or on the success labels used for evaluation; the reported success rates are environment-defined for Meta-World and classifier-defined for the real-world tasks. The per-task similarity thresholds and view weights in Table I are hyperparameters selected for the evaluation tasks, which is a tuning and generalization concern rather than a circularity, because the reward formula is not derived from the success signal it is later compared against. The only substantive discrepancy found is evidential, not circular: Contribution 3 and the abstract attribute the 29.0% real-world improvement to 'without environmental rewards', while Section V-E and Figure 11 evaluate TeViR+ (Equation 6), which explicitly includes the sparse reward r_spar; the pure no-environment-reward TeViR (Equation 5) is tested only in the Meta-World section. That mismatch affects whether the headline claim is supported, not whether the derivation reduces to its inputs. No uniqueness theorem, ansatz, or load-bearing self-citation is used to force the reward design; the self-citations in the references are routine and not load-bearing. Consequently there is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The method is not a parameter-free derivation: it depends on hand-set per-task thresholds and view weights, and on the unvalidated premise that latent cosine similarity tracks task progress. These are the main costs the reader pays beyond the pretrained diffusion model.

free parameters (4)
  • similarity threshold theta = 0.7 to 0.9 per task in Table I
    The progress reward in Equation (4) is a binary comparison against theta, and Table I assigns a different theta to each task. No principled selection method is given.
  • view weights w_left, w_top, w_close = Per-task triples in Table I, e.g. {0.5, 0.8, 0.4} for button-press-topdown-v2
    Section V-F says the weights are adjusted to align with the specific requirements of each task, so the reward function is tuned on the evaluation tasks.
  • progress reward scale alpha = 0.125
    Global hyperparameter chosen by hand; the authors state it is suitable for all selected tasks, but no sensitivity study is provided.
  • number of generated frames H = 8
    Fixed design choice for all experiments; no ablation tests other horizon lengths.
assumptions (4)
  • domain assumption Expert videos from scripted policies or human teleoperation are accurate demonstrations of successful task execution and are sufficient to train the text-to-video model.
    Section V-A: 30 expert videos per task are collected via scripted policy or human teleoperation, and the diffusion model is trained on them. If these videos are not representative, the generated rewards are wrong.
  • domain assumption Cosine similarity in VQ-GAN latent space between the current observation and a generated frame is a valid, roughly monotonic proxy for task progress.
    Equations (2)-(4) define the distance and progress rewards entirely through this cosine similarity. The paper does not validate that latent similarity tracks true task stage.
  • domain assumption The future video generated from the initial frame and language description remains a relevant expert trajectory for all later states in the rollout.
    Section IV-C compares every current observation against the one generated sequence from z0; if the agent diverges from that plan, the comparison may be meaningless.
  • domain assumption The real-world sparse reward produced by a pre-trained ResNet classifier is a correct success detector.
    Section V-A states that real-world sparse rewards are generated by the classifier; errors in the classifier propagate into TeViR+ and the evaluation metric.
invented entities (1)
  • Reached image progress counter M_t
    purpose: Tracks the farthest generated frame the agent has matched, enforcing that intermediate task steps are completed in order when computing the progress reward.
    Defined in Section IV-C and Figure 3; there is no external validation that M_t corresponds to true task stage, and its update depends on the hand-set threshold theta.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TeViR: Text-to-Video Reward with Diffusion Models for Efficient Reinforcement Learning." pith.science (2026). https://pith.science/paper/3C24JPFC

@misc{pith2026250519769,
  author       = {Pith},
  title        = {Pith review of: TeViR: Text-to-Video Reward with Diffusion Models for Efficient Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3C24JPFC}},
  note         = {Machine review of arXiv:2505.19769}
}
read the original abstract

Developing scalable and generalizable reward engineering for reinforcement learning (RL) is crucial for creating general-purpose agents, especially in the challenging domain of robotic manipulation. While recent advances in reward engineering with Vision-Language Models (VLMs) have shown promise, their sparse reward nature significantly limits sample efficiency. This paper introduces TeViR, a novel method that leverages a pre-trained text-to-video diffusion model to generate dense rewards by comparing the predicted image sequence with current observations. Experimental results across 11 complex robotic tasks demonstrate that TeViR outperforms traditional methods leveraging sparse rewards and other state-of-the-art (SOTA) methods, achieving better sample efficiency and performance without ground truth environmental rewards. TeViR's ability to efficiently guide agents in complex environments highlights its potential to advance reinforcement learning applications in robotic manipulation.

Figures

Figures reproduced from arXiv: 2505.19769 by the authors.

Figure 1
Figure 1. Overall framework of TeViR. The text-to-video diffusion model takes the initial RGB observation of the environment and a textual goal description as [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the 3 view selected for the Meta-World environment and the network architecture of our text-to-video diffusion model. We chose [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Progress calculation illustration. For each image [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Trajectory visualizations of ’window-close-v2’ from [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Performances of the different reward engineering methods with the environmental sparse reward on 8 tasks from Meta-World benchmark. Raw Sparse [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Overview of all real-world experimental tasks, including a) Pick Carrot [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Performances of the different reward engineering methods without the environmental feedback on 8 tasks from Meta-World benchmark. RoboCLIP, [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Performances of TeViR with added noise on 2 selected tasks from [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 7
Figure 7. Figure 7: The experimental results demonstrate that our method achieves outstanding performance across all tasks in the ab￾sence of the ground truth environmental sparse rewards. This finding underscores the robustness and effectiveness of our approach in complex robot manipulat…
Figure 10
Figure 10. Figure 10: Trajectory returns of TeViR on real-world ”Open Drawer” task. Only [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 12
Figure 12. Figure 12: Ablation studies for validating the necessity of multi-view observa [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Ablation studies for validating the necessity of each reward [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 49 canonical work pages

  1. [2]

    Reinforcement learning control of a flexible two-link manipulator: An experimental investiga- tion,

    W. He, H. Gao, C. Zhou, C. Yang, and Z. Li, “Reinforcement learning control of a flexible two-link manipulator: An experimental investiga- tion,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 12, pp. 7326–7336, 2021

    work page 2021

  2. [3]

    A hierarchical deep reinforcement learning framework for 6-dof ucav air-to-air combat,

    J. Chai, W. Chen, Y . Zhu, Z.-X. Yao, and D. Zhao, “A hierarchical deep reinforcement learning framework for 6-dof ucav air-to-air combat,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 53, no. 9, pp. 5417–5429, 2023

    work page 2023

  3. [4]

    Conrft: A reinforced fine-tuning method for vla models via consistency policy,

    Y . Chen, S. Tian, S. Liu, Y . Zhou, H. Li, and D. Zhao, “Conrft: A reinforced fine-tuning method for vla models via consistency policy,” in Robotics: Science and Systems RSS, 2025

    work page 2025

  4. [5]

    End-to-end robotic rein- forcement learning without reward engineering,

    A. Singh, L. Yang, C. Finn, and S. Levine, “End-to-end robotic rein- forcement learning without reward engineering,” inRobotics: Science and Systems XV, 2019

    work page 2019

  5. [6]

    Deep learning in robotics: Survey on model structures and training strategies,

    A. I. K ´aroly, P. Galambos, J. Kuti, and I. J. Rudas, “Deep learning in robotics: Survey on model structures and training strategies,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 1, pp. 266–279, 2021

    work page 2021

  6. [7]

    Deep reinforcement learning-based automatic exploration for navigation in unknown environment,

    H. Li, Q. Zhang, and D. Zhao, “Deep reinforcement learning-based automatic exploration for navigation in unknown environment,”IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 6, pp. 2064–2076, 2020

    work page 2020

  7. [8]

    Reward design with language models,

    M. Kwon, S. M. Xie, K. Bullard, and D. Sadigh, “Reward design with language models,” inInternational Conference on Learning Represen- tations ICLR, 2023

    work page 2023

  8. [9]

    A survey of inverse reinforcement learning: Challenges, methods and progress,

    S. Arora and P. Doshi, “A survey of inverse reinforcement learning: Challenges, methods and progress,”Artif. Intell., vol. 297, p. 103500, 2021

    work page 2021

Show all 55 references
  1. [10]

    Concrete problems in ai safety,

    D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Man ´e, “Concrete problems in ai safety,”arXiv preprint:1606.06565, 2016

    2016 arXiv

  2. [11]

    VIP: towards universal visual reward and representation via value-implicit pre-training,

    Y . J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V . Kumar, and A. Zhang, “VIP: towards universal visual reward and representation via value-implicit pre-training,” inInternational Conference on Learning Representations ICLR, 2023

    2023

  3. [13]

    Robot fine-tuning made easy: Pre-training rewards and policies for autonomous real-world reinforcement learning,

    J. Yang, M. S. Mark, B. Vu, A. Sharma, J. Bohg, and C. Finn, “Robot fine-tuning made easy: Pre-training rewards and policies for autonomous real-world reinforcement learning,” inIEEE International Conference on Robotics and Automation, ICRA, 2024

    2024

  4. [14]

    Roboclip: One demonstration is enough to learn robot policies,

    S. Sontakke, J. Zhang, S. M. R. Arnold, K. Pertsch, E. Biyik, D. Sadigh, C. Finn, and L. Itti, “Roboclip: One demonstration is enough to learn robot policies,” inAnnual Conference on Neural Information Processing Systems NIPS, 2023

    2023

  5. [15]

    RL-VLM-F: reinforcement learning from vision language foundation model feedback,

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. Erickson, “RL-VLM-F: reinforcement learning from vision language foundation model feedback,” inInternational Conference on Machine Learning, ICML, 2024

    2024

  6. [16]

    Video diffusion models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,”Advances in Neural Information Processing Systems, vol. 35, pp. 8633–8646, 2022

    2022

  7. [17]

    Imagen video: High definition video generation with diffusion models,

    J. Ho, W. Chan, C. Saharia, J. Whang, R. Gao, A. Gritsenko, D. P. Kingma, B. Poole, M. Norouzi, D. J. Fleetet al., “Imagen video: High definition video generation with diffusion models,”arXiv preprint:2210.02303, 2022

    2022 arXiv

  8. [18]

    Learning universal policies via text-guided video generation,

    Y . Du, S. Yang, B. Dai, H. Dai, O. Nachum, J. Tenenbaum, D. Schuur- mans, and P. Abbeel, “Learning universal policies via text-guided video generation,” inAnnual Conference on Neural Information Processing Systems NIPS, 2023

    2023

  9. [19]

    Learning to act from actionless videos through dense correspondences,

    P. Ko, J. Mao, Y . Du, S. Sun, and J. B. Tenenbaum, “Learning to act from actionless videos through dense correspondences,” inInternational Conference on Learning Representations ICLR, 2024

    2024

  10. [20]

    Learning interactive real-world simu- lators,

    S. Yang, Y . Du, S. K. S. Ghasemipour, J. Tompson, L. P. Kaelbling, D. Schuurmans, and P. Abbeel, “Learning interactive real-world simu- lators,” inInternational Conference on Learning Representations ICLR, 2024

    2024

  11. [21]

    Any-point trajectory modeling for policy learning,

    C. Wen, X. Lin, J. I. R. So, K. Chen, Q. Dou, Y . Gao, and P. Abbeel, “Any-point trajectory modeling for policy learning,” inRobotics: Science and Systems, 2024

    2024

  12. [22]

    Foundation reinforcement learning: towards embodied generalist agents with foundation prior assistance,

    W. Ye, Y . Zhang, M. Wang, S. Wang, X. Gu, P. Abbeel, and Y . Gao, “Foundation reinforcement learning: towards embodied generalist agents with foundation prior assistance,”arXiv preprint:2310.02635, 2023

    2023 arXiv

  13. [23]

    Video prediction models as rewards for reinforcement learning,

    A. Escontrela, A. Adeniji, W. Yan, A. Jain, X. B. Peng, K. Goldberg, Y . Lee, D. Hafner, and P. Abbeel, “Video prediction models as rewards for reinforcement learning,” inAnnual Conference on Neural Informa- tion Processing Systems NIPS, 2023

    2023

  14. [24]

    Diffusion reward: Learning rewards via conditional video diffusion,

    T. Huang, G. Jiang, Y . Ze, and H. Xu, “Diffusion reward: Learning rewards via conditional video diffusion,” inEuropean Computer Vision Association ECCV, 2024

    2024

  15. [25]

    Stabilizing diffusion model for robotic control with dynamic programming and transition feasibility,

    H. Li, Y . Zhang, H. Wen, Y . Zhu, and D. Zhao, “Stabilizing diffusion model for robotic control with dynamic programming and transition feasibility,”IEEE Transactions on Artificial Intelligence, vol. 1, no. 01, pp. 1–11, 2024

    2024

  16. [26]

    Decision making with visualizations: a cognitive framework across disciplines,

    L. M. Padilla, S. H. Creem-Regehr, M. Hegarty, and J. K. Stefanucci, “Decision making with visualizations: a cognitive framework across disciplines,”Cognitive research: principles and implications, vol. 3, pp. 1–25, 2018

    2018

  17. [27]

    MAT: Morphological adaptive trans- former for universal morphology policy learning,

    B. Li, H. Li, Y . Zhu, and D. Zhao, “MAT: Morphological adaptive trans- former for universal morphology policy learning,”IEEE Transactions on Cognitive and Developmental Systems, pp. 1–12, 2024

    2024

  18. [28]

    Boosting continuous control with consistency policy,

    Y . Chen, H. Li, and D. Zhao, “Boosting continuous control with consistency policy,” inInternational Conference on Autonomous Agents and Multiagent Systems AAMAS, 2024

    2024

  19. [29]

    Proximal policy optimization with policy feedback,

    Y . Gu, Y . Cheng, C. L. P. Chen, and X. Wang, “Proximal policy optimization with policy feedback,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 52, no. 7, pp. 4600–4610, 2022

    2022

  20. [30]

    Interactive language: Talking to robots in real time,

    C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence, “Interactive language: Talking to robots in real time,”IEEE Robotics and Automation Letters, 2023

    2023

  21. [31]

    The surprising effectiveness of representation learning for visual imitation,

    J. Pari, N. M. M. Shafiullah, S. P. Arunachalam, and L. Pinto, “The surprising effectiveness of representation learning for visual imitation,” inRobotics: Science and Systems XVIII, 2022

    2022

  22. [32]

    Watch and act: Learning robotic manipulation from visual demonstration,

    S. Yang, W. Zhang, R. Song, J. Cheng, H. Wang, and Y . Li, “Watch and act: Learning robotic manipulation from visual demonstration,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 53, no. 7, pp. 4404–4416, 2023

    2023

  23. [33]

    Apprenticeship learning via inverse reinforce- ment learning,

    P. Abbeel and A. Y . Ng, “Apprenticeship learning via inverse reinforce- ment learning,” inInternational Conference on Machine Learning ICML, 2004

    2004

  24. [34]

    Guided cost learning: Deep inverse optimal control via policy optimization,

    C. Finn, S. Levine, and P. Abbeel, “Guided cost learning: Deep inverse optimal control via policy optimization,” inInternational Conference on Machine Learning ICML, 2016

    2016

  25. [35]

    XIRL: cross-embodiment inverse reinforcement learning,

    K. Zakka, A. Zeng, P. Florence, J. Tompson, J. Bohg, and D. Dwibedi, “XIRL: cross-embodiment inverse reinforcement learning,” inConfer- ence on Robot Learning CORL, 2021

    2021

  26. [36]

    Unsupervised perceptual rewards for imitation learning,

    P. Sermanet, K. Xu, and S. Levine, “Unsupervised perceptual rewards for imitation learning,” inRobotics: Science and Systems XIII, 2017

    2017

  27. [37]

    Learning generalizable robotic reward functions from

    A. S. Chen, S. Nair, and C. Finn, “Learning generalizable robotic reward functions from ”in-the-wild” human videos,” inRobotics: Science and Systems XVII, 2021

    2021

  28. [38]

    Can pre-trained text-to-image models generate visual goals for reinforcement learning?

    J. Gao, K. Hu, G. Xu, and H. Xu, “Can pre-trained text-to-image models generate visual goals for reinforcement learning?” inAnnual Conference on Neural Information Processing Systems NIPS, 2023. 13

    2023

  29. [39]

    Language instructed reinforcement learning for human-ai coordination,

    H. Hu and D. Sadigh, “Language instructed reinforcement learning for human-ai coordination,” inInternational Conference on Machine Learning ICML, 2023

    2023

  30. [40]

    Language to rewards for robotic skill synthesis,

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K. Lee, M. G. Arenas, H. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, B. Ichter, T. Xiao, P. Xu, A. Zeng, T. Zhang, N. Heess, D. Sadigh, J. Tan, Y . Tassa, and F. Xia, “Language to rewards for robotic skill synthesis,” inConference on R...

    2023

  31. [41]

    Robogen: Towards unleashing infinite data for automated robot learning via generative simulation,

    Y . Wang, Z. Xian, F. Chen, T. Wang, Y . Wang, K. Fragkiadaki, Z. Erickson, D. Held, and C. Gan, “Robogen: Towards unleashing infinite data for automated robot learning via generative simulation,” inInternational Conference on Machine Learning, ICML, 2024

    2024

  32. [42]

    Eureka: Human-level reward design via coding large language models,

    Y . J. Ma, W. Liang, G. Wang, D. Huang, O. Bastani, D. Jayaraman, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human-level reward design via coding large language models,” inInternational Conference on Learning Representations ICLR, 2024

    2024

  33. [43]

    Vision-language models as success detectors,

    Y . Du, K. Konyushkova, M. Denil, A. Raju, J. Landon, F. Hill, N. de Fre- itas, and S. Cabi, “Vision-language models as success detectors,” in Conference on Lifelong Learning Agents, 2023

    2023

  34. [44]

    Guiding pretraining in reinforcement learning with large language models,

    Y . Du, O. Watkins, Z. Wang, C. Colas, T. Darrell, P. Abbeel, A. Gupta, and J. Andreas, “Guiding pretraining in reinforcement learning with large language models,” inInternational Conference on Machine Learn- ing ICML, 2023

    2023

  35. [45]

    Vision-language models are zero-shot reward models for reinforcement learning,

    J. Rocamonde, V . Montesinos, E. Nava, E. Perez, and D. Lindner, “Vision-language models are zero-shot reward models for reinforcement learning,” inInternational Conference on Learning Representations ICLR, 2024

    2024

  36. [46]

    LIV: language-image representations and rewards for robotic control,

    Y . J. Ma, V . Kumar, A. Zhang, O. Bastani, and D. Jayaraman, “LIV: language-image representations and rewards for robotic control,” in International Conference on Machine Learning, ICML, ser. Proceedings of Machine Learning Research, vol. 202. PMLR, 2023, pp. 23 301– 23 320

    2023

  37. [47]

    Llmscenario: Large language model driven scenario generation,

    C. Chang, S. Wang, J. Zhang, J. Ge, and L. Li, “Llmscenario: Large language model driven scenario generation,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 54, no. 11, pp. 6581– 6594, 2024

    2024

  38. [48]

    Zero-shot robotic manipulation with pretrained image- editing diffusion models,

    K. Black, M. Nakamoto, P. Atreya, H. Walke, C. Finn, A. Kumar, and S. Levine, “Zero-shot robotic manipulation with pretrained image- editing diffusion models,”arXiv preprint: 2310.10639, 2023

    2023 arXiv

  39. [49]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” inInternational Conference on Learning Representations ICLR, 2021

    2021

  40. [50]

    Taming transformers for high- resolution image synthesis,

    P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high- resolution image synthesis,” inIEEE Conference on Computer Vision and Pattern Recognition CVPR, 2021

    2021

  41. [51]

    Exploration by random network distillation,

    Y . Burda, H. Edwards, A. J. Storkey, and O. Klimov, “Exploration by random network distillation,” inInternational Conference on Learning Representations, ICLR, 2019

    2019

  42. [52]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,

    T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine, “Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,” inConference on Robot Learning, 2019

    2019

  43. [53]

    Learning transferable visual models from natural language supervi- sion,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervi- sion,” inInternational Conference on Machine Learning ICML, 2021

    2021

  44. [54]

    Mastering visual continuous control: Improved data-augmented reinforcement learning,

    D. Yarats, R. Fergus, A. Lazaric, and L. Pinto, “Mastering visual continuous control: Improved data-augmented reinforcement learning,” inInternational Conference on Learning Representations ICLR, 2022

    2022

  45. [55]

    Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning,

    J. Luo, C. Xu, J. Wu, and S. Levine, “Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning,”arXiv preprint:2410.21845, 2024

    2024 arXiv

  46. [56]

    Con- trolvideo: Training-free controllable text-to-video generation,

    Y . Zhang, Y . Wei, D. Jiang, X. Zhang, W. Zuo, and Q. Tian, “Con- trolvideo: Training-free controllable text-to-video generation,” inInter- national Conference on Learning Representations ICLR, 2024

    2024

  47. [57]

    Ldr: Learning discrete representa- tion to improve noise robustness in multiagent tasks,

    Y . Fu, Y . Zhu, J. Chai, and D. Zhao, “Ldr: Learning discrete representa- tion to improve noise robustness in multiagent tasks,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 55, no. 1, pp. 513–525, 2025

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.