Pith. sign in

REVIEW 3 major objections 5 minor 28 references

A robot can improve its own behavior tree through closed-loop verbal critique, without any gradient-based reinforcement learning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 17:36 UTC pith:3FP7KC53

load-bearing objection The framework is a sensible combination and the hardware demo is real, but the empirical support is too thin to back the headline claims. the 3 major comments →

arxiv 2603.22169 v3 pith:3FP7KC53 submitted 2026-03-23 cs.RO

VersualRL: Closed-Loop Verbal Reinforcement Learning with Visual Execution Feedback for Task-Level Robot Planning

Dmitrii Plotnikov , Iaroslav Kolomiets , Dmitrii Maliukov , Dmitrij Kosenkov , Daniia Zinniatullina , Artem Trandofilov , Georgii Gazaryan , Kirill Bogatikov
show 4 more authors
Timofei Kozlov Mikhail Konenkov Miguel Altamirano Cabrera Dzmitry Tsetserukou
This is my paper
classification cs.RO
keywords Verbal reinforcement learningBehavior TreesRobot task planningVision-language modelsExecution feedbackClosed-loop planningMobile manipulationLegibility
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's central project is to show that a physical robot's task policy can be improved through a closed loop of language critiques: a vision-language model watches the robot work and writes structured natural-language feedback keyed to failures, while a language-model actor edits the robot's Behavior Tree plan accordingly. This replaces the usual gradient-based reinforcement learning with discrete, human-readable plan edits. The authors validate this on a real mobile robot performing a multi-stage navigation and manipulation task under execution uncertainty, reporting measurable score improvements across episodes. If the claim holds, it would mean that task-level robot learning can be transparent, hardware-aware, and free of simulators and reward engineering.

Core claim

The paper claims that verbal reinforcement learning—using structured natural-language feedback as the learning signal—can improve executable Behavior Trees on a physical robot without any gradient updates. The critic outputs not only a textual diagnosis but also an alarm severity score and a confidence value, so human oversight can be triggered; the actor then makes discrete edits to the tree. Experiments show that pure scalar scores are insufficient under uncertainty, that causal attribution by the critic matters more than raw perceptual accuracy, and that adding structured symbolic inputs accelerates convergence. The authors also observe that the actor implicitly compensates for critic bli

What carries the argument

The actor–critic loop is the core mechanism. The VLM critic operates in three modes (Initial, Intermediate, Final), analyzing visual snapshots together with Behavior Tree execution traces and emitting a natural-language critique, an alarm score in [0,1], and a confidence value; scores of 0.5 or higher (or confidence below 0.3) alert a human operator. The LLM actor receives the critique, the real score, and a bounded context of task definition, environment, node library, and authoring rules, then proposes discrete, syntactic modifications to the Behavior Tree. This loop, iterated over episodes, performs closed-loop policy refinement at the symbolic level.

Load-bearing premise

The loop assumes every relevant fault is visible in the critic's snapshot window and can be causally linked to recent Behavior Tree events; if a failure has delayed consequences or leaves no visible trace, the actor's edit may target the wrong cause.

What would settle it

Run episodes with a planted delayed-effect fault—e.g., a block that is picked correctly but later topples because of an earlier mis-grasp that is no longer visible when the toppling occurs. If the critic never sees the original mis-grasp, the loop should fail to correct it and the Behavior Tree should not improve on that fault, whereas a version with a longer observation horizon or explicit root-cause tracing should correct it.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Numerical reward signals alone do not distinguish hardware errors from planning errors; structured verbal feedback is needed for stable improvement.
  • A critic's ability to attribute faults to specific Behavior Tree nodes matters more than its raw detection accuracy.
  • Providing structured symbolic perceptual inputs (e.g., block orientation information) reduces ambiguity and speeds up convergence.
  • The actor implicitly adapts to the critic's perceptual limits, keeping learning stable even when feedback is imperfect.
  • Because the policy is edited directly in a symbolic tree structure, every update remains human-interpretable and hardware-specific, without simulators.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the critic is indeed the bottleneck, the framework could be pushed further by coupling it with uncertainty-aware fusion of multiple critics or by explicitly modeling faults that fall outside the snapshot window.
  • The same loop could be adapted to multi-robot settings where one shared critic guides several actors, or where a human intermittently injects corrective feedback into the loop.
  • A natural stress test: plant faults whose consequences appear several steps later, and observe whether the actor receives misleading feedback—this should reveal whether the snapshot-window assumption is the key limit.
  • The architecture suggests that any improvement in symbolic policy expressiveness (richer node libraries) would translate directly into better final policies once the critic is reliable.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes VersualRL, a closed-loop verbal reinforcement learning framework for task-level robot planning. A frozen Vision-Language Model (VLM) critic observes visual snapshots and Behavior Tree (BT) execution traces and outputs structured natural-language critique together with alarm and confidence scalars; a frozen Large Language Model (LLM) actor then edits the BT symbolically. No gradient-based optimization is used. The framework is validated on a real mobile manipulator performing a warehouse-style navigation and manipulation task under execution uncertainty. Five critic/input configurations are compared across five field configurations with ten episodes each. The paper claims explainable policy improvement, closed-loop adaptation to execution failures, and reliable physical deployment.

Significance. If validated, the framework is a useful contribution: it offers an interpretable, hardware-in-the-loop alternative to gradient-based RL for task-level planning, explicitly separates causal diagnosis (VLM critic) from symbolic policy modification (LLM actor), and demonstrates a real-robot deployment with frozen foundation models. The algorithmic description and the qualitative trajectory/BT evolution examples (Figs. 8–9) are illustrative and helpful. However, the empirical evidence as presented is not yet sufficient to substantiate the headline claims, because the central learning-curve results are means without error bars or statistical tests, and the fine-tuned 3B comparison is confounded by an undisclosed task-specific dataset. These issues are load-bearing for the paper's main conclusion that critic feedback produces measurable policy improvement.

major comments (3)
  1. [Section V / Fig. 4] The central claim that critic feedback yields policy improvement is based on mean episode-score curves averaged over five field configurations, with ten episodes each. The paper reports only means and no standard deviations, confidence intervals, per-configuration curves, or significance tests. With n=5 independent configurations, the visible separation between critic configurations and the no-critic baseline, and between Gemini with and without BlockInfo, could plausibly arise from run-to-run noise. Since the paper repeatedly states that results 'show' improvement (Abstract, Section V), the statistical support must be established. Please provide per-environment results, error bars, and appropriate paired or repeated-measures tests on final scores or learning-curve summaries.
  2. [Section V, Qwen fine-tuned comparison] The claim that 'domain adaptation is more critical than model scale' is confounded. The Qwen2.5-VL-3B critic was 'fine-tuned on our structured execution-feedback dataset,' but this dataset is not described, quantified, or released. The 7B model is used off-the-shelf. The observed 3B advantage may therefore reflect the fine-tuning data and task-specific supervision rather than domain adaptation or scale per se. To support the conclusion, the dataset composition, annotation scheme, size, and training split must be disclosed, and ideally the same fine-tuning protocol should be applied to the 7B model as a control.
  3. [Section IV.C / Algorithm 1] The experimental protocol does not state whether actor memory H is reset between field configurations or between the five configurations, nor whether the same initial BT0 is used throughout. Algorithm 1 initializes H once before the episode loop, but the experiments execute 'ten training episodes' per field configuration and then move to another configuration. If H persists across configurations, the five configurations are not independent and the effective sample size is less than five; if H is reset, that should be stated explicitly. This ambiguity affects the validity of any statistical interpretation of the learning curves.
minor comments (5)
  1. [Section V, Figures] All four figures (4–7) would benefit from error bars or shaded confidence regions. Currently they show only mean trajectories, which makes it impossible to judge variability.
  2. [Section V] The sentence about the no-critic baseline, 'the learning process a safety-aware feedback channel,' is grammatically incomplete. Please revise.
  3. [Section VI] The explicit acknowledgment that the framework assumes faults are visible within the critic's snapshot-based window and can be causally linked to recent BT events is welcome. However, the earlier language of 'explicit causal feedback' (Abstract, Section III) is stronger than this limitation permits; consider moderating the terminology.
  4. [Throughout] Model naming is inconsistent: 'Qwen2.5 VL 7B Instruct' vs 'Qwen2.5-VL-7B' and 'Gemini 3 Pro Preview' vs 'Gemini-3-Pro-Preview.' Please standardize.
  5. [Algorithm 1] The termination condition 'while episode not terminated' is not formalized. Please state whether termination is automatic (e.g., robot returns to finish area) or requires human intervention, as this affects reproducibility.

Circularity Check

0 steps flagged

No significant circularity: the framework's central claims are empirical comparisons, not derivations that reduce to their inputs.

full rationale

The paper's central claim—that a VLM critic providing structured verbal feedback enables closed-loop policy improvement—is supported by an empirical comparison of critic configurations against a no-critic baseline, not by a derivation from the framework's own definitions. The actor updates the Behavior Tree using environment-derived RealScore and critic feedback, while evaluation uses a separate rule-based episodic score; using the same task score as reward and evaluation metric is standard practice in RL and does not make the result circular because the score originates from the environment, not from the model. The fine-tuned Qwen-3B critic is explicitly described as 'fine-tuned on our structured execution-feedback dataset,' so its advantage over the untuned 7B model is presented as a fitted-model comparison, not as an unfitted prediction; the undisclosed dataset is a reproducibility concern, not a circular step. The alarm thresholds and severity rubrics in Section III.C are hand-tuned system components that condition the critic's outputs, but they do not by construction determine the measured score improvement or the relative ordering of configurations. The only self-citation, [16] (LLM-MARS), appears in a list of related LLM-planning systems and is not load-bearing; no uniqueness theorem or prior result by the authors is invoked to force the framework's design. The acknowledged limitations in the Discussion—that faults must be visible in the critic's snapshot window and causally linkable to recent BT events—are honest scope boundaries, not circular reasoning. The skeptic's concern about missing error bars and significance tests is a statistical-evidence threat to the headline empirical claim, but it is a correctness risk, not circularity. Overall, no load-bearing step reduces to its own inputs, so the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The framework rests on hand-crafted scoring, alarm thresholds, and domain assumptions about critic reliability and BT-edit ability. The only new entity is the 'VRL' algorithm composition itself, which is built from existing models.

free parameters (4)
  • Episode score weights = Table I: -1 per second, -20 time limit, +10 per box, +10 batch bonus, -5 misplace, -10 outside, +20 final position
    The task score function is hand-authored; all learning curves and conclusions depend on this metric.
  • Alarm thresholds = s>=0.5 actionable, s>0.7 severe, c<0.3 notify
    Hand-set in Section III-C; these determine human alarm rate and actor update trigger.
  • Episode budget = N=10
    Ten training episodes per field configuration; chosen without stated justification, affects convergence claims.
  • BlockInfo symbolic input = block color/orientation provided to actor in one ablation
    An extra hand-provided perceptual abstraction; without it, convergence is slower, but it acts as an external oracle-like input.
axioms (5)
  • domain assumption The rule-based score R faithfully reflects task success
    Used to drive updates and evaluate; no validation of the score's alignment with human judgment is presented.
  • domain assumption The VLM critic's scalar alarm/confidence are meaningful across models and episodes
    No calibration against human labels is shown; the severity rubric is ordinal and hand-defined.
  • domain assumption The LLM actor can reliably parse critic feedback and apply syntactically valid BT edits
    Central to the loop; no verification of edit validity or edit-error rate is provided.
  • domain assumption Relevant faults are visible in the critic's snapshot window and causally linked to recent BT events
    Stated in Discussion as a limitation: delayed or hidden faults are not handled, yet the method relies on this for trustworthy edits.
  • ad hoc to paper Five field configurations with fixed block orientations are representative of execution uncertainty
    No rationale is given for choosing these five configurations; the small, non-random sample limits generalization claims.

pith-pipeline@v1.3.0-alltime-deepseek · 8941 in / 10193 out tokens · 103010 ms · 2026-08-02T17:36:01.093541+00:00 · methodology

0 comments
read the original abstract

We introduce VersualRL, a closed-loop framework for task-level robot planning that uses visual execution feedback to iteratively refine executable Behavior Trees through structured natural-language critique. VersualRL combines a vision-language model critic, which analyzes visual observations and Behavior Tree execution traces, with a large language model actor that performs discrete and interpretable policy updates. During physical deployment, both foundation models remain frozen, while the executed Behavior Tree is updated at the symbolic level without online gradient-based policy optimization. This enables transparent reasoning, explicit causal feedback, and human-interpretable policy evolution. We validate VersualRL on a real mobile robot performing a multi-stage navigation and manipulation task under execution uncertainty. Experimental results show that the framework supports explainable policy improvements, closed-loop adaptation to execution failures, and successful deployment on physical robotic systems.

Figures

Figures reproduced from arXiv: 2603.22169 by Artem Trandofilov, Daniia Zinniatullina, Dmitrii Maliukov, Dmitrii Plotnikov, Dmitrij Kosenkov, Dzmitry Tsetserukou, Georgii Gazaryan, Iaroslav Kolomiets, Kirill Bogatikov, Miguel Altamirano Cabrera, Mikhail Konenkov, Timofei Kozlov.

Figure 1
Figure 1. Figure 1: System architecture. from the physical world. As a result, they struggle to reliably adapt to real-world execution failures. To address these limitations, we propose a closed-loop Verbal Reinforcement Learning (VRL) framework for task￾level robotic planning. In VRL, task policies are represented as executable Behavior Trees and iteratively refined through structured natural-language feedback. An LLM-based … view at source ↗
Figure 2
Figure 2. Figure 2: Hardware architecture of the mobile robot: (1) three-wheeled [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Layout of the experimental environment: (1) start/finish area, (2) [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Mean Accuracy across environments. We first analyze the episodic score dynamics, which directly reflect task-level performance and successful policy refinement. Our fine-tuned Qwen2.5-VL-3B model demonstrates more consistent and pronounced score improvement compared to the larger Qwen2.5-VL-7B model. Although the 3B variant has fewer parameters, it was fine-tuned on our structured execution-feedback datase… view at source ↗
Figure 6
Figure 6. Figure 6: Mean Confidence across environments. This metric measures how consistently the critic identifies real execution problems observed in the scene. Our fine-tuned Qwen2.5-VL-3B model demonstrates higher and more stable accuracy compared to the larger Qwen2.5-VL-7B model. Despite having fewer parameters, the domain-adapted 3B critic better captures task-specific failure modes, indicating that fine-tuning improv… view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative BT evolution under perceptual uncertainty. [PITH_FULL_IMAGE:figures/full_fig_p006_9.png] view at source ↗
Figure 8
Figure 8. Figure 8: Episode of the Qwen2.5-VL-3B (fine-tuned) critic. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 5 linked inside Pith

  1. [1]

    Decision-making for path planning of mobile robots under uncertainty: A review of belief-space planning simplifications,

    V . Malathi, P. Sreedharan, R. P. R, V . A. Kumar, A. L. Sadasivan, G. Udupa, L. Pastorelli, and A. Troppina, “Decision-making for path planning of mobile robots under uncertainty: A review of belief-space planning simplifications,”Robotics, vol. 14, no. 9, p. 127, 2025

    2025

  2. [2]

    Thrun, W

    S. Thrun, W. Burgard, and D. Fox,Probabilis- tic Robotics. MIT Press, 2005. [Online]. Available: https://mitpress.mit.edu/9780262201629/probabilistic-robotics/

    arXiv 2005

  3. [3]

    Behavior tree learning for robotic task planning through monte carlo dag search over a formal grammar,

    E. Scheide, G. Best, and G. A. Hollinger, “Behavior tree learning for robotic task planning through monte carlo dag search over a formal grammar,” inProc. IEEE Int. Conf. Robotics and Automation (ICRA), 2021, pp. 4837–4843

    2021

  4. [4]

    How behavior trees modularize hybrid control systems and generalize sequential behavior composi- tions, the subsumption architecture, and decision trees,

    M. Colledanchise and P. ¨Ogren, “How behavior trees modularize hybrid control systems and generalize sequential behavior composi- tions, the subsumption architecture, and decision trees,”IEEE Trans. Robotics, vol. 33, no. 2, pp. 372–389, 2017

    2017

  5. [5]

    Concrete problems in ai safety,

    D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Man ´e, “Concrete problems in ai safety,” 2016, arXiv:1606.06565

    Pith/arXiv arXiv 2016

  6. [6]

    Do as i can, not as i say: Grounding language in robotic affordances,

    B. Ichter, A. Brohan, Y . Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang,et al., “Do as i can, not as i say: Grounding language in robotic affordances,” inProc. 6th Conf. Robot Learning (CoRL), ser. Proceedings of Machine Learning Research, vol. 205, 2023, pp. 287–318

    2023

  7. [7]

    Code as policies: Language model programs for embodied control,

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for embodied control,” inProc. IEEE Int. Conf. Robotics and Automation (ICRA), 2023, pp. 9493–9500

    2023

  8. [8]

    Progprompt: Program generation for situated robot task planning using large language models,

    I. Singh, V . Blukis, A. Mousavian,et al., “Progprompt: Program generation for situated robot task planning using large language models,”Autonomous Robots, vol. 47, pp. 999–1012, 2023

    2023

  9. [9]

    Autogpt+p: Affordance- based task planning using large language models,

    T. Birr, C. Pohl, A. Younes, and T. Asfour, “Autogpt+p: Affordance- based task planning using large language models,” inProc. Robotics: Science and Systems (RSS), 2024

    2024

  10. [10]

    Re- flexion: Language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Re- flexion: Language agents with verbal reinforcement learning,” inProc. 37th Int. Conf. Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, 2023, p. Art. no. 377

    2023

  11. [11]

    Re- act: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, I. Shafran, K. R. Narasimhan, and Y . Cao, “Re- act: Synergizing reasoning and acting in language models,” inProc. NeurIPS 2022 Foundation Models for Decision Making Workshop, 2022

    2022

  12. [12]

    Self- refine: Iterative refinement with self-feedback,

    A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y . Yang, S. Gupta, B. P. Majumder, K. Hermann, S. Welleck, A. Yazdanbakhsh, and P. Clark, “Self- refine: Iterative refinement with self-feedback,” inProc. 37th Int. Conf. Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, 2023, p. Art....

    2023

  13. [13]

    V oxposer: Composable 3d value maps for robotic manipulation with language models,

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei, “V oxposer: Composable 3d value maps for robotic manipulation with language models,” inProc. 7th Annu. Conf. Robot Learning (CoRL),

  14. [14]

    Task and motion planning with large language models for object rearrangement,

    Y . Ding, X. Zhang, C. Paxton, and S. Zhang, “Task and motion planning with large language models for object rearrangement,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), Detroit, MI, USA, 2023, pp. 2086–2092

    2023

  15. [15]

    Innermonologue: Embodied reasoning through planning with language models,

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. J. R. Tompson, I. Mordatch, Y . Chebotar,et al., “Innermonologue: Embodied reasoning through planning with language models,” in Proc. Conf. Robot Learning (CoRL), 2022, to appear. [Online]. Available: https://innermonologue.github.io/

    2022

  16. [16]

    Llm-mars: Large language model for behavior tree generation and nlp-enhanced dialogue in multi-agent robot systems,

    A. Lykov, M. Dronova, N. Naglov, M. Litvinov, S. Satsevich, A. Bazhenov, V . Berman, A. Shcherbak, and D. Tsetserukou, “Llm-mars: Large language model for behavior tree generation and nlp-enhanced dialogue in multi-agent robot systems,” 2023, arXiv:2312.09348

    Pith/arXiv arXiv 2023

  17. [17]

    Vision- language model-driven scene understanding and robotic object ma- nipulation,

    S. Liu, J. Zhang, R. X. Gao, X. V . Wang, and L. Wang, “Vision- language model-driven scene understanding and robotic object ma- nipulation,” inProc. IEEE 20th Int. Conf. Automation Science and Engineering (CASE), 2024, pp. 21–26

    2024

  18. [18]

    Large language models for robotics: Opportunities, challenges, and perspectives,

    J. Wang, E. Shi, H. Hu, C. Ma, Y . Liu, X. Wang, Y . Yao, X. Liu, B. Ge, and S. Zhang, “Large language models for robotics: Opportunities, challenges, and perspectives,”Journal of Automation and Intelligence, vol. 4, no. 1, pp. 52–64, 2025

    2025

  19. [19]

    V oyager: An open-ended embodied agent with large language models,

    G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. Fan, and A. Anandkumar, “V oyager: An open-ended embodied agent with large language models,”Trans. Mach. Learn. Res., 2024

    2024

  20. [20]

    Btgenbot: Behavior tree generation for robotic tasks with lightweight llms,

    R. A. Izzo, G. Bardaro, and M. Matteucci, “Btgenbot: Behavior tree generation for robotic tasks with lightweight llms,” inProc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), 2024, pp. 9684– 9690

    2024

  21. [21]

    Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. Erickson, “Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,” inProc. 41st Int. Conf. Machine Learning (ICML), Vienna, Austria, 2024, p. Art. no. 2112

    2024

  22. [22]

    Vision-language models are zero-shot reward models for reinforce- ment learning,

    J. Rocamonde, V . Montesinos, E. Nava, E. Perez, and D. Lindner, “Vision-language models are zero-shot reward models for reinforce- ment learning,” inProc. 12th Int. Conf. Learning Representations (ICLR), 2024

    2024

  23. [23]

    Varp: Reinforcement learning from vision- language model feedback with agent regularized preferences,

    A. Singh, A. Bhaskar, P. Yu, S. Chakraborty, R. Dasyam, A. S. Bedi, and P. Tokekar, “Varp: Reinforcement learning from vision- language model feedback with agent regularized preferences,” 2025, arXiv:2503.13817

    Pith/arXiv arXiv 2025

  24. [24]

    Clip-motion: Learning reward func- tions for robotic actions using consecutive observations,

    X. Dang and S. Edelkamp, “Clip-motion: Learning reward func- tions for robotic actions using consecutive observations,” 2025, arXiv:2311.03485

    Pith/arXiv arXiv 2025

  25. [25]

    Learning reward for robot skills using large language models via self-alignment,

    Y . Zeng, Y . Mu, and L. Shao, “Learning reward for robot skills using large language models via self-alignment,” inProc. 41st Int. Conf. Machine Learning (ICML), Vienna, Austria, 2024, p. Art. no. 2408

    2024

  26. [26]

    Affordance- guided reinforcement learning via visual prompting,

    O. Y . Lee, A. Xie, K. Fang, K. Pertsch, and C. Finn, “Affordance- guided reinforcement learning via visual prompting,” inProc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), Hangzhou, China, 2025, pp. 2352–2359

    2025

  27. [27]

    Task success is not enough: Investigating the use of video-language models as behavior critics for catching undesirable agent behaviors,

    L. Guan, Y . Zhou, D. Liu, Y . Zha, H. B. Amor, and S. Kambhampati, “Task success is not enough: Investigating the use of video-language models as behavior critics for catching undesirable agent behaviors,” inProc. First Conf. Language Modeling, 2024

    2024

  28. [2023]

    Available: https://doi.org/10.48550/arXiv.2307.05973

    [Online]. Available: https://doi.org/10.48550/arXiv.2307.05973

    Pith/arXiv arXiv doi:10.48550/arxiv.2307.05973