Pith. sign in

REVIEW 4 cited by

Vision-Language Models as a Source of Rewards

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.09187 v3 pith:PRWSEKXD submitted 2023-12-14 cs.LG

Kate Baumli , Satinder Baveja , Feryal Behbahani , Harris Chan , Gheorghe Comanici , Sebastian Flennerhag , Maxime Gazeau , Kristian Holsheimer
show 19 more authors
Dan Horgan Michael Laskin Clare Lyle Hussain Masoom Kay McKinney Volodymyr Mnih Alexander Neitz Dmitry Nikulin Fabio Pardo Jack Parker-Holder John Quan Tim Rocktäschel Himanshu Sahni Tom Schaul Yannick Schroecker Stephen Spencer Richie Steigerwald Luyu Wang Lei Zhang
This is my paper · ORCID
classification cs.LG
keywords agentsgoalsrewardsmodelsvisualachievementbuildinggeneralist
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning. A key limiting factor for building generalist agents with RL has been the need for a large number of reward functions for achieving different goals. We investigate the feasibility of using off-the-shelf vision-language models, or VLMs, as sources of rewards for reinforcement learning agents. We show how rewards for visual achievement of a variety of language goals can be derived from the CLIP family of models, and used to train RL agents that can achieve a variety of language goals. We showcase this approach in two distinct visual domains and present a scaling trend showing how larger VLMs lead to more accurate rewards for visual goal achievement, which in turn produces more capable RL agents.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning

    cs.RO 2026-03 conditional novelty 6.5 of 10

    A video-language model with per-timestep spatiotemporal CoT and dense progress prediction can serve as the sole reward for zero-shot online robot RL on 24 unseen manipulation tasks.

  2. FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making

    cs.RO 2025-07 conditional novelty 6.0 of 10

    FOUNDER maps foundation-model embeddings of text or video prompts into world-model goal states and rewards policies by predicted temporal distance to those goals, improving reward-free multi-task offline control.

  3. RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills

    cs.RO 2025-06 conditional novelty 6.0 of 10

    RobotSmith autonomously designs, 3D-prints, and uses task-specific tools for robotic manipulation, raising task success from 2.8% (no tool) to 50% in simulation.

  4. Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

    cs.RO 2026-07 conditional novelty 5.0 of 10

    A survey organizing progress reward modeling for robot learning into interface, method, and data/evaluation layers, with a four-family method taxonomy and a cautionary split between progress fidelity and downstream utility.

Pith tools