Pith. sign in

REVIEW 8 cited by

Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.13816 v2 pith:OAXHWHKX submitted 2024-10-17 cs.RO cs.LG

Mitsuhiko Nakamoto , Oier Mees , Aviral Kumar , Sergey Levine This is my paper · ORCID
classification cs.ROcs.LG
keywords policiesdifferentperformancerobotictrainedvalueapproachdata
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large, general-purpose robotic policies trained on diverse demonstration datasets have been shown to be remarkably effective both for controlling a variety of robots in a range of different scenes, and for acquiring broad repertoires of manipulation skills. However, the data that such policies are trained on is generally of mixed quality -- not only are human-collected demonstrations unlikely to perform the task perfectly, but the larger the dataset is, the harder it is to curate only the highest quality examples. It also remains unclear how optimal data from one embodiment is for training on another embodiment. In this paper, we present a general and broadly applicable approach that enhances the performance of such generalist robot policies at deployment time by re-ranking their actions according to a value function learned via offline RL. This approach, which we call Value-Guided Policy Steering (V-GPS), is compatible with a wide range of different generalist policies, without needing to fine-tune or even access the weights of the policy. We show that the same value function can improve the performance of five different state-of-the-art policies with different architectures, even though they were trained on distinct datasets, attaining consistent performance improvement on multiple robotic platforms across a total of 12 tasks. Code and videos can be found at: https://nakamotoo.github.io/V-GPS

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

    cs.LG 2026-04 accept novelty 7.0 of 10

    WAV self-improves action-conditioned world models by cycle-consistent verification of state plausibility and sparse action reachability, doubling sample efficiency and lifting policy reward by over 22% on nine tasks.

  2. SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning

    cs.RO 2026-03 conditional novelty 6.5 of 10

    A video-language model with per-timestep spatiotemporal CoT and dense progress prediction can serve as the sole reward for zero-shot online robot RL on 24 unseen manipulation tasks.

  3. DREAMSTEER: Latent World Models Can Steer VLA Policies During Deployment Without Any Finetuning

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A frozen VLA plus latent world-model rollouts and a value model can raise real-robot OOD manipulation success from 23.75% to 66.25% without any target-environment finetuning.

  4. EVE: A Generator-Verifier System for Generative Policies

    cs.RO 2025-12 conditional novelty 6.0 of 10

    Zero-shot VLM verifiers, ensembled and fused via guided diffusion, improve frozen generative robot policies' success rates by 1-2 percentage points on simulated manipulation tasks.

  5. Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distribution

    cs.RO 2025-08 conditional novelty 6.0 of 10

    Latent Policy Barrier improves behavior-cloned visuomotor policies by using a latent dynamics model trained on expert and rollout data to guide actions back toward in-distribution expert states.

  6. LLM-as-a-Verifier: A General-Purpose Verification Framework

    cs.AI 2026-07 conditional novelty 5.0 of 10

    Expecting over scoring-token logits yields continuous, scalable verification that improves agent trajectory selection and dense RL rewards across coding, robotics, and medical benchmarks.

  7. Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models

    cs.RO 2025-09 conditional novelty 5.0 of 10

    ARFM adaptively adjusts a scaling factor in the flow-matching loss so that offline RL advantage signals are preserved while gradient variance is controlled, improving VLA robot policy fine-tuning.

  8. Steering Robots with Inference-Time Interactions

    cs.RO 2025-06 conditional novelty 4.0 of 10

    Frozen imitation policies can be steered at inference time via user interactions, with a diffusion-sampling method and a constraint-enforcing framework that provides formal task guarantees.

Pith tools