Pith. sign in

REVIEW 3 major objections 5 minor 61 references

Adaptation of Generalist Robot Policies with Minimal Data

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that one successful demonstration, followed by autonomous practice, is enough for a pre-trained generalist robot policy to learn a new task reliably.

desk verdict A well-run simulation recipe for one-demo adaptation that overclaims on the real-world 'autonomous' evidence; the missing success-detection detail in the hardware section is the thing to fix. read the letter →

arxiv 2608.11363 v1 pith:N4ZKFQUV submitted 2026-08-11 cs.RO cs.LG

Shreyas Kowshik , Sreyas Venkataraman , Leo Wang , Niharika Pant , Max Simchowitz , Aviral Kumar This is my paper · ORCID
classification cs.ROcs.LG
keywords minimal-dataadaptationvision-language-actionmodelsresidualreinforcementlearningsingle-demonstrationbehaviorcloningpolicy-agnosticRLsparse-rewardonlinerobotpolicygeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that minimal-data adaptation is tractable: a pre-trained generalist robot policy can learn a new task from as little as one successful demonstration followed by autonomous online interaction. It argues that one demonstration works not because it teaches the full task, but because it anchors the policy near task-relevant behavior, after which sparse-reward value-based reinforcement learning on a lightweight residual module supplies the missing control corrections. Across LIBERO-Long and RoboCasa, the proposed recipe, MiDAS, recovers strong task performance from one demonstration, outperforming baselines that only sharpen or steer the imitated policy, and it transfers to a real bimanual platform over roughly six hours of autonomous practice. If correct, this shows that today's vision-language-action policies can turn a single human demonstration into a bootstrap for autonomous improvement.

What carries the argument

The load-bearing object is the residual policy $\pi^{\mathrm{res}}_\theta(\cdot \mid s_t, a^{\mathrm{base}}_t)$: a lightweight MLP that directly predicts the executed action chunk, conditioned jointly on the frozen VLA's state representation and on the action chunk proposed by the frozen base policy, trained with policy-agnostic RL (PA-RL). PA-RL improves the policy by sampling candidate chunks, refining the top candidates with gradient ascent on an ensemble Q-function, and distilling the refined actions back into the parametric policy. Around this core sit two stabilizers: an offline warmup phase that initializes the residual actor to reproduce base proposals and calibrates the critic on the demonstration buffer, and success balancing, which mixes successful trajectories into every actor and critic batch so that sparse successes are not drowned out by failures.

What would settle it

Audit the real-hardware practice phase: take the roughly 350 trajectories collected over 5-6 hours on the YAM platform, have an independent human label each as success or failure, and compare against the automatic signal that filled the success buffer; if agreement is not near-perfect, or if the pipeline in fact required a human to score rollouts, the claim of autonomous adaptation is not supported. In simulation the boundary is already visible: the RoboCasa dishwasher task, where the one-demonstration warm start achieves 0% success, also yields 0% for MiDAS, so demonstrating whether any interaction budget can lift a 0% warm-start task would fix the precise limits of the one-demonstration claim.

Watch

Extended reading notes

Core claim

MiDAS is a two-stage recipe. Stage I fine-tunes a pre-trained vision-language-action policy on the single demonstration with behavior cloning, producing a policy that coarsely follows the task instruction but completes it only rarely. Stage II freezes that policy and trains a lightweight residual actor-critic on top of its representations using policy-agnostic RL: at each state the residual policy samples candidate action chunks conditioned on the frozen policy's proposal, refines the best ones by gradient ascent on a learned Q-function, and distills them back. Three ingredients make this work: an offline warmup that calibrates the critic and residual actor on the demonstration and a few base-policy rollouts; success balancing that oversamples the rare successful trajectories; and direct prediction of the executed action, which lets the policy move beyond the base policy's effective action support. The paper reports that value-based RL finds corrective actions that imitation-sharpening and steering baselines cannot, that frozen VLA representations are what make online value learning sample-efficient, and that the adapted policy inherits visual and language robustness from the frozen backbone while state shifts requiring genuinely new behaviors remain out of reach unless the reset distribution is expanded by a curriculum.

Load-bearing premise

The load-bearing premise is that the real-hardware practice phase is genuinely autonomous: the algorithm's success buffer and value estimates require every rollout to be scored correctly by the system itself, yet the paper never describes how success is detected on the YAM platform, only that objects are reset around demonstration locations, so if a human judged or relabeled rollouts the autonomous-adaptation claim would weaken.

Editorial extensions

If this is right

  • A single demonstration can replace the tens to hundreds of demonstrations that prior residual-RL and offline-to-online methods assume, at least for tasks inside the base policy's behavioral repertoire.
  • Because the backbone stays frozen and only a small residual module is trained, the compute, memory, and stability cost of online adaptation is small enough to run on a real robot for a few hours.
  • Imitation-only improvement (filtering successful rollouts) and in-support steering (perturbing the diffusion noise) plateau where value-based RL succeeds, implying that the missing behavior after one demonstration often lies outside the imitated policy's action support.
  • The frozen backbone's invariances transfer: adapted policies keep most of their in-distribution performance under visual appearance changes and language paraphrases, so online RL adds control without sacrificing observation-level generalization.
  • Shifts that require a genuinely new manipulation strategy (object swaps or category changes) defeat the adapted policy, but a reset-distribution curriculum during online RL closes most of the position-generalization gap without additional demonstrations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Success balancing implies a boundary condition the paper states only implicitly: MiDAS bootstraps only when the one-demonstration policy succeeds at least occasionally, since the success buffer is seeded and replenished by actual successful rollouts; the reported dishwasher task, which stays at zero success, marks that boundary.
  • A natural measure of what one demonstration covers is the number of behavioral modes it anchors: the object-swap results suggest each demonstration covers one spatial arrangement and grasp strategy, so a testable extension is to probe the adapted policy under systematic permutations of object positions to count how many modes a single demonstration supports.
  • The per-joint trust-region added on the real robot hints that constraining how far the residual may deviate from the frozen proposal is load-bearing on hardware; ablating that constraint on the YAM platform would isolate whether direct action prediction alone is stable or whether the constraint is what keeps training from drifting.
  • The paper's split between observation shifts and state shifts suggests a division of labor for future systems: pretraining supplies invariance, demonstrations supply behavioral modes, and autonomous interaction supplies robustness within a mode; one could test this by giving the same recipe one demonstration per mode and checking whether performance is additive across modes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces minimal-data adaptation (MDA), in which a pre-trained VLA policy must learn a task from one or few demonstrations followed by autonomous online interaction, and proposes MiDAS: Stage I behavior-clones the base policy on K demonstrations (LoRA on the VLM, full action head), then Stage II freezes the policy and trains a lightweight residual actor-critic with value-based RL using an offline warmup, success-buffer oversampling, and PA-RL-style action refinement. Experiments on LIBERO-Long (10 tasks) and RoboCasa (4 tasks, with one excluded from the average) report that MiDAS reaches high success with K=1 and outperforms BC, DSRL, and Filtered BC, while DICE-RL is competitive on LIBERO. Additional analyses attribute the gains to pretrained representations, coarse task anchoring from BC, and value-guided corrections beyond the base policy's action support. Real-world bimanual YAM experiments report improvements from 40% to 67% (T1) and 27% to 80% (T2) over 5-6 hours, and the paper claims this is the first demonstration of reliable robot policy adaptation from a single task demonstration.

Significance. If the claims hold, the paper offers a practically attractive recipe: a frozen VLA plus a lightweight residual policy and value function can convert a single demonstration into robust task performance, and the ablation isolating frozen VLA representations is a useful scientific finding. The simulation study is a strength, with 3 seeds, 50 evaluation rollouts per task, held-out evaluation, and a strong residual-RL baseline (DICE-RL) whose LIBERO performance is comparable to MiDAS. The paper is also transparent about several limitations, including the LoadDishwasher failure, object-swap collapse, and the pick-and-place scope discussed in Section 8. No circularity is evident: results are measured on held-out rollouts against external baselines rather than being baked into the method by construction. However, the real-world 'autonomous' claim currently rests on an unspecified success-detection mechanism, the real-world quantitative evidence is statistically thin, and the RoboCasa headline average omits a task on which MiDAS fails. These issues are fixable but are load-bearing for the strongest claims in the abstract.

major comments (3)
  1. [Algorithm 1 / Appendix I] The headline real-world claim of 'reliable robot policy adaptation' depends on autonomous success detection, but the manuscript does not specify how success is determined during the 5-6 hour online phase. Algorithm 1 populates the success buffer using 'successful(·)' rollouts (lines 4 and 14), and Eq. (3.1) uses the resulting sparse reward for the TD target; both require a reliable binary success signal at every episode. Appendix I describes resets and evaluation but only states that target objects are reset around the demonstration locations and that ID/OOD rollouts are collected; it never says whether success is judged by an automated vision/force check, a learned classifier, or a human. If a human labels episodes or intervenes during practice, the 'autonomous interaction' claim is overstated; if the detector is noisy, the success-balancing updates (Eq. D.1) amplify labeling errors. Please specify the mechanism, report its accuracy if learned, and revise the abstract and Section 4 claims accordingly.
  2. [Table 3 / Appendix I] The real-world success rates in Table 3 are point estimates over 15 rollouts with no confidence intervals or significance tests. On T1 the reported improvement from 40% to 67% is not statistically significant at conventional levels (a Fisher exact test on 15 rollouts per condition gives p>0.1), so the quantitative real-world evidence for 'reliable' adaptation is essentially T2 alone. Because the abstract's novelty claim is framed around a single demonstration, the paper should report per-condition counts and intervals, provide a statistical test, and ideally collect more rollouts, or temper the real-world part of the claim.
  3. [Table 1 (RoboCasa) and Appendix C.1] The claim 'Across LIBERO-Long and RoboCasa, MiDAS recovers strong task performance' overstates the RoboCasa results as presented. The fourth task, Cup + Bowl→Dishwasher + Close, achieves 0% success for MiDAS and all baselines and is excluded from the RoboCasa average with only a dagger footnote; Appendix C.1 later concedes that this task 'shows the limitation of our proposed approach.' Excluding a failed task from the headline average without a stated exclusion criterion makes the cross-benchmark claim difficult to evaluate. Please either include the task in the average (reporting 0% for MiDAS on it), justify the exclusion in the main text, or soften the claim to the tasks where MiDAS succeeds.
minor comments (5)
  1. [Throughout] The baseline name is spelled inconsistently as 'DiceRL' (Table 1), 'DICE-RL' (Sections 4 and Appendix C.1), and 'Dice-RL' (Appendix C.1); please unify the notation.
  2. [Eq. (4.1) and Appendix D.1] Equation (4.1) writes the residual actor as tanh(𝒩(μ_θ, σ_θ²)), but the sampling/reparameterization and the density used for the PA-RL distillation of refined actions are not defined; please specify how tanh-squashed samples are generated and how the distillation loss treats the output distribution.
  3. [Appendix I] The sentence 'The residual policy is queried every 30 steps within each 60-step base action chunk after which is new chunk is sampled' contains a typo and an ambiguity: clarify whether the residual is queried at step 30 and again at step 60, and how the executed chunk is composed.
  4. [Table 2 and Figure 5] The claim that MiDAS reaches actions outside the base policy's support is supported by one critical state and a min-distance metric; the 'higher Q*' annotation in Figure 5 is suggestive but not measured. Reporting critic values and success outcomes across multiple critical states would strengthen this mechanistic claim.
  5. [Tables 14, 22, 23, 25] The recipe has several important hyperparameters (ρ_succ, N, M, warmup steps, update-to-data ratios, LoRA rank/alpha) with no sensitivity analysis; a brief sensitivity study, or at least a statement of which knobs were not tuned, would make the 'simple recipe' claim more robust.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: MiDAS is an empirical recipe evaluated on held-out rollouts against external baselines; the self-cited PA-RL component is a subroutine, not the evidence for the paper's claims.

full rationale

MiDAS does not contain a derivation in which an output is equivalent to an input by construction. Stage I behavior-clones a pre-trained VLA on K demonstrations; Stage II freezes the backbone and trains a residual actor-critic with sparse-reward online RL. Success rates in Tables 1, 3, and the perturbation tables are measured on held-out rollouts and compared with external baselines (Filtered BC, DSRL, DICE-RL, flow-matching BC), so the headline numbers are not baked into the method. The claim that few-demo BC "recovers task-level behavior but not reliable control" is an empirical observation (Fig. 2, Table 1), not a redefinition. The claim that value-based RL "expands the base policy's action support" is supported by the Min-Dist evidence in Table 2, an independent measurement, not by the PA-RL citation. PA-RL [27] has overlapping authorship (Aviral Kumar), but it is used as an off-the-shelf policy-extraction subroutine with its own external prior publication; the paper's central adaptation claim does not reduce to any theorem or fitted value from that citation. The real-world "autonomous" assertion has a genuine empirical gap (Appendix I describes resets but not the success-detection mechanism used during the 5-6 hour online phase), and the T1 point estimates over 15 rollouts are statistically weak, but these are correctness and measurement concerns, not circularity under the rubric. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported to forbid alternatives, and no known result is merely renamed.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

MiDAS rests on standard RL machinery plus hand-tuned hyperparameters that are not derived from first principles. The central empirical claims do not require invented physical entities; the main additional burden is the assumption that frozen VLA representations support value learning and that autonomous success detection is reliable in the real-world setup.

free parameters (7)
  • Success buffer ratio rho_succ = 0.20
    Chosen by hand in Table 14; controls how strongly successful transitions are oversampled during critic and actor updates, which is critical when initial success is 2-5%.
  • PA-RL candidate and elite counts (N, M) = 16 candidates, 8 elites
    Chosen in Table 14; determines exploration breadth and how many refined actions are distilled into the residual actor.
  • BC warmup steps = 10,000
    Chosen in Table 14 and Algorithm 1; calibrates the residual actor and critic before online RL begins, and the paper shows removing it destabilizes training.
  • Update-to-data ratios = 30 critic updates, 10 actor updates per environment step
    Chosen in Table 14; balances learning speed against stability and was not derived from any principle.
  • Action chunk length C = 10 in simulation, 60 in real world
    Chosen in Appendices A and I; defines the chunked action MDP and affects temporal abstraction and credit assignment.
  • Real-world trust-region radius = 0.2 rad
    Chosen in Table 25; limits how far refined joint actions can move from the base policy proposal, and is specific to the real-hardware experiments.
  • LoRA rank and alpha = rank 32, alpha 32
    Chosen in Table 15; controls the capacity of Stage I behavior cloning and is not fitted to any external benchmark.
assumptions (6)
  • domain assumption Sparse binary reward with -1 per step and 0 on success is a sufficient objective for solving the target task consistently.
    Section 3 defines this reward; if the binary success signal does not accurately detect task completion, the entire RL objective is misaligned.
  • standard math TD learning with an ensemble critic and target networks provides reliable value estimates for policy improvement under function approximation.
    Section 3 and Appendix D rely on standard TD and target-network machinery without convergence guarantees.
  • domain assumption The frozen VLA backbone of pi0.5 provides action-aligned visual representations that enable sample-efficient value learning.
    Section 5.2 and Figure 3 support this empirically, but the method inherits all representation quality from pi0.5 and would fail with weaker representations.
  • domain assumption One-demo behavior cloning places the base policy close enough to task success that rollouts contain near-success states for critic calibration.
    Section 4 warmup and Section 5.1; if the base policy's success rate is too low or its rollouts are too far from success, the success buffer and critic cannot bootstrap.
  • ad hoc to paper In the real-world setup, success and failure are detected reliably and automatically during online training.
    Appendix I describes resets and evaluation but never states how episode success is determined on hardware, which is required for the success buffer and sparse-reward critic.
  • domain assumption Evaluation rollouts and seeds are representative of the intended deployment distribution.
    Tables 1 and 3 use 50 or 15 rollouts per task; the real-world evaluation has no stated seed count or variance, so the assumption is weaker there.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptation of Generalist Robot Policies with Minimal Data." pith.science (2026). https://pith.science/paper/N4ZKFQUV

@misc{pith2026260811363,
  author       = {Pith},
  title        = {Pith review of: Adaptation of Generalist Robot Policies with Minimal Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N4ZKFQUV}},
  note         = {Machine review of arXiv:2608.11363}
}
read the original abstract

A central goal in robot learning is to move beyond task-specific human data collection toward robots that improve through autonomous interaction. Yet fully autonomous learning remains difficult with current policies: sparse rewards and weak zero-shot exploration make it unlikely that a robot will discover successful behavior from scratch. We study minimal-data adaptation, a regime in which a pre-trained robot policy must learn a new task from as little as one demonstration followed by autonomous online interaction. This setting serves as the closest tractable proxy for fully autonomous improvement, allowing us to study whether minimal human guidance can bootstrap autonomous learning and what algorithmic ingredients make it feasible. We build MiDAS, a simple offline-to-online RL recipe that first anchors a pre-trained VLA to the target task with behavior cloning on single/few demonstrations, then improves it through value-based online RL on a residual policy parameterization. Across LIBERO and RoboCasa, MiDAS recovers strong task performance from as little as one demonstration, substantially outperforming baselines and generalizing beyond demonstrated conditions. We further evaluate MiDAS on a bimanual YAM platform. Starting from a fragile low-success policy obtained from a single demonstration, MiDAS improves its robustness and learns new successful behaviors over ~6 hours of online interaction. To the best of our knowledge, this is the first demonstration of reliable robot policy adaptation from a single task demonstration.

Figures

Figures reproduced from arXiv: 2608.11363 by the authors.

Figure 1
Figure 1. Overview of minimal-data adaptation. A single demonstration is sparse and underdetermined on its own, but pre-trained VLA representations turn it into a useful anchor for adaptation. Few-demo behavior cloning narrows the search space to a task-relevant region around the demonstration, and online residual RL then expands local success by learning value-guided corrections on top of the warm-started base policy. We stu… view at source ↗
Figure 2
Figure 2. Few-demo BC recovers task-level behavior, but not fine-grained control. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Performance of different representations in online adaptation. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Performance of different task representations for behavior cloning Qualitative rollouts in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: MiDAS reaches actions outside the effective sup￾port of 𝜋 𝐾 base. Filtered BC and DSRL remain near samples from 𝜋 𝐾 base and fail; MiDAS selects an action in a disjoint region with higher 𝑄 * and succeeds. 3) Value-based RL learns fine-grained control actions beyond th…
Figure 6
Figure 6. Figure 6: Representative real-world rollouts. Rollouts illustrating task progress on the knife-and-donut task (top) and the block placement task (bottom). Key Findings • Expert demonstrations induce coarse task-solving behavior. Behavior cloning on minimal demon￾strations instil…
Figure 7
Figure 7. Figure 7: Performance of BC and online RL under observation and state shifts. Observation Shifts (Visual/Language): Pre-trained representations preserve performance under visual and language changes. State Shifts (Swap/Shape/Object): Online RL improves robustness to moderate shi…
Figure 8
Figure 8. Figure 8: Curriculum closes the gap. Progressively widening the reset distribution during online RL re￾covers most of the position-generalization gap with 𝜋 50 base without additional demos. (Q3) Broader coverage, either from demonstrations or curriculum, expands the generalizat…
Figure 9
Figure 9. Figure 9: Libero-Long Learning Curves for MiDAS with baseline comparison. Solid line shows mean across 3 seeds and shaded region denotes standard deviation [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: RoboCasa-365 Learning Curves for MiDAS with baseline comparison. Solid line shows mean across 3 seeds and shaded region denotes standard deviation. Each task is instantiated in a specific kitchen layout and style that is fixed across training and evaluation. The envir…
Figure 11
Figure 11. Figure 11: MiDAS ablations on both-mokapots task random within the specified radius from their default positions. The policy advances to the next stage once its evaluation success rate on the current perturbation level exceeds a threshold of 0.6. On the final stage, training con…
Figure 12
Figure 12. Figure 12: (a) 𝜋 1 𝑟𝑙 covers a narrow region of state space around its warm-start demo; 𝜋 50 base visits a much broader distribution. (b) Success degrades with displacement magnitude; distilling 𝜋 1 𝑟𝑙 rollouts into the base policy does not close the gap with 𝜋 50 base. of the p…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 22 canonical work pages

  1. [1]

    From imitation to refinement – residual rl for precise assembly, 2024

    Lars Ankile, Anthony Simeonov, Idan Shenfeld, Marcel Torne, and Pulkit Agrawal. From imitation to refinement – residual rl for precise assembly, 2024. URLhttps://arxiv.org/abs/2407.16677

    arXiv 2024

  2. [2]

    Residual off-policy rl for finetuning behavior cloning policies, 2025

    LarsAnkile, ZhenyuJiang, RockyDuan, GuanyaShi, PieterAbbeel, andAnushaNagabandi. Residual off-policy rl for finetuning behavior cloning policies, 2025. URLhttps://arxiv.org/abs/2509. 19301

    work page 2025

  3. [3]

    Ball, Laura Smith, Ilya Kostrikov, and Sergey Levine

    Philip J. Ball, Laura Smith, Ilya Kostrikov, and Sergey Levine. Efficient online reinforcement learning with offline data, 2023. URLhttps://arxiv.org/abs/2302.02948

    arXiv 2023

  4. [4]

    URLhttps://arxiv.org/abs/2410.24164

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky.𝜋0: A visio...

    arXiv 2026

  5. [5]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control, 2023

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gon- zalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnik...

    arXiv 2023

  6. [6]

    Rt-1: Robotics transformer for real-world control at scale, 2023

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, ...

    arXiv 2023

  7. [7]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott G...

    work page 1901

  8. [8]

    Emerging properties in self-supervised vision transformers, 2021

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers, 2021. URLhttps: //arxiv.org/abs/2104.14294

    arXiv 2021

Show all 61 references
  1. [9]

    Causal confusion in imitation learning

    Pim De Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. Advances in Neural Information Processing Systems, 32, 2019

    2019

  2. [10]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors,Proceedings of the 2019 Conference of the North American Chapter of ...

    2019 doi

  3. [11]

    Expo: Stable reinforcement learning with expressive policies, 2026

    Perry Dong, Qiyang Li, Dorsa Sadigh, and Chelsea Finn. Expo: Stable reinforcement learning with expressive policies, 2026. URLhttps://arxiv.org/abs/2507.07986

    2026 arXiv

  4. [12]

    Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojciech Zaremba

    Yan Duan, Marcin Andrychowicz, Bradly C. Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojciech Zaremba. One-shot imitation learning, 2017. URLhttps://arxiv. org/abs/1703.07326

    2017 arXiv

  5. [13]

    One-shot visual imitation learning via meta-learning, 2017

    Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. One-shot visual imitation learning via meta-learning, 2017. URLhttps://arxiv.org/abs/1709.04905

    2017 arXiv

  6. [14]

    A minimalist approach to offline reinforcement learning,

    Scott Fujimoto and Shixiang Shane Gu. A minimalist approach to offline reinforcement learning,

  7. [15]

    Addressing function approximation error in actor-critic methods, 2018

    Scott Fujimoto, Herke van Hoof, and David Meger. Addressing function approximation error in actor-critic methods, 2018. URLhttps://arxiv.org/abs/1802.09477

    2018 arXiv

  8. [16]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 2018

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 2018. URLhttps:// arxiv.org/abs/1801.01290

    2018 arXiv

  9. [17]

    Teach a robot to fish: Versatile imitation from one minute of demonstrations, 2023

    Siddhant Haldar, Jyothish Pari, Anant Rai, and Lerrel Pinto. Teach a robot to fish: Versatile imitation from one minute of demonstrations, 2023. URLhttps://arxiv.org/abs/2303.01497

    2023 arXiv

  10. [18]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URLhttps: //arxiv.org/abs/2106.09685

    2021 arXiv

  11. [19]

    Imitation bootstrapped reinforcement learning, 2024

    Hengyuan Hu, Suvir Mirchandani, and Dorsa Sadigh. Imitation bootstrapped reinforcement learning, 2024. URLhttps://arxiv.org/abs/2311.02198

    2024 arXiv

  12. [20]

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin Le...

    2025 arXiv

  13. [21]

    Fine-tuningvision-language-actionmodels: Optimizing speed and success, 2025

    MooJinKim, ChelseaFinn, andPercyLiang. Fine-tuningvision-language-actionmodels: Optimizing speed and success, 2025. URLhttps://arxiv.org/abs/2502.19645

    2025 arXiv

  14. [22]

    Offline reinforcement learning with implicit q- learning, 2021

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q- learning, 2021. URLhttps://arxiv.org/abs/2110.06169

    2021 arXiv

  15. [23]

    Pre-training for robots: Offline rl enables learning new tasks from a handful of trials, 2023

    Aviral Kumar, Anikait Singh, Frederik Ebert, Mitsuhiko Nakamoto, Yanlai Yang, Chelsea Finn, and Sergey Levine. Pre-training for robots: Offline rl enables learning new tasks from a handful of trials, 2023. URLhttps://arxiv.org/abs/2210.05178

    2023 arXiv

  16. [24]

    Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning, 2023. URLhttps://arxiv.org/ abs/2306.03310

    2023 arXiv

  17. [25]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,

  18. [26]

    Serl: A software suite for sample-efficient robotic reinforcement learning, 2025

    Jianlan Luo, Zheyuan Hu, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek Gupta, and Sergey Levine. Serl: A software suite for sample-efficient robotic reinforcement learning, 2025. URLhttps://arxiv.org/abs/2401.16013

    2025 arXiv

  19. [27]

    Policy agnostic rl: Offline rl and online rl fine-tuning of any class and backbone, 2024

    Max Sobol Mark, Tian Gao, Georgia Gabriela Sampaio, Mohan Kumar Srirama, Archit Sharma, Chelsea Finn, and Aviral Kumar. Policy agnostic rl: Offline rl and online rl fine-tuning of any class and backbone, 2024. URLhttps://arxiv.org/abs/2412.06685

    2024 arXiv

  20. [28]

    Cal-ql: Calibrated offline rl pre-training for efficient online fine-tuning,

    Mitsuhiko Nakamoto, Yuexiang Zhai, Anikait Singh, Max Sobol Mark, Yi Ma, Chelsea Finn, Aviral Kumar, and Sergey Levine. Cal-ql: Calibrated offline rl pre-training for efficient online fine-tuning,

  21. [29]

    Robocasa365: A large- scale simulation framework for training and benchmarking generalist robots, 2026

    Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddukuri, and Yuke Zhu. Robocasa365: A large- scale simulation framework for training and benchmarking generalist robots, 2026. URLhttps: //arxiv.org/abs/2603.04356

    2026

  22. [30]

    Keypoint action tokens enable in-context imitation learning in robotics, 2024

    Norman Di Palo and Edward Johns. Keypoint action tokens enable in-context imitation learning in robotics, 2024. URLhttps://arxiv.org/abs/2403.19578

    2024 arXiv

  23. [31]

    Much ado about noising: Dispelling the myths of generative robotic control, 2026

    Chaoyi Pan, Giri Anantharaman, Nai-Chieh Huang, Claire Jin, Daniel Pfrommer, Chenyang Yuan, Frank Permenter, Guannan Qu, Nicholas Boffi, Guanya Shi, and Max Simchowitz. Much ado about noising: Dispelling the myths of generative robotic control, 2026. URLhttps://arxiv.org/ abs/...

    2026

  24. [32]

    Ogpo: Sample efficient full-finetuning of generative control policies, 2026

    Sarvesh Patil, Mitsuhiko Nakamoto, Manan Agarwal, Shashwat Saxena, Jesse Zhang, Giri Anantharaman, Cleah Winston, Chaoyi Pan, Douglas Chen, Nai-Chieh Huang, Zeynep Temel, Oliver Kroemer, Sergey Levine, Abhishek Gupta, Hongkai Dai, Paarth Shah, and Max Sim- chowitz. Ogpo: Sampl...

    2026 arXiv

  25. [33]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URLhttps://arx...

    2021 arXiv

  26. [34]

    Ren, Justin Lidard, Lars L

    Allen Z. Ren, Justin Lidard, Lars L. Ankile, Anthony Simeonov, Pulkit Agrawal, Anirudha Majumdar, Benjamin Burchfiel, Hongkai Dai, and Max Simchowitz. Diffusion policy policy optimization, 2024. URLhttps://arxiv.org/abs/2409.00588

    2024 arXiv

  27. [36]

    From prior to pro: Efficient skill mastery via distribution contractive rl finetuning, 2026

    Zhanyi Sun and Shuran Song. From prior to pro: Efficient skill mastery via distribution contractive rl finetuning, 2026. URLhttps://arxiv.org/abs/2603.10263

    2026 arXiv

  28. [37]

    Octo: An open-source generalist robot policy, 2024

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yunliang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo...

    2024 arXiv

  29. [38]

    Instant policy: In-context imitation learning via graph diffusion,

    Vitalis Vosylius and Edward Johns. Instant policy: In-context imitation learning via graph diffusion,

  30. [39]

    Steering your diffusion policy with latent space reinforcement learning, 2025

    Andrew Wagenmaker, Mitsuhiko Nakamoto, Yunchu Zhang, Seohong Park, Waleed Yagoub, Anusha Nagabandi, Abhishek Gupta, and Sergey Levine. Steering your diffusion policy with latent space reinforcement learning, 2025. URLhttps://arxiv.org/abs/2506.15799

    2025 arXiv

  31. [40]

    Rl token: Bootstrapping online rl with vision-language-action models, 2026

    Charles Xu, Jost Tobias Springenberg, Michael Equi, Ali Amin, Adnan Esmail, Sergey Levine, and Liyiming Ke. Rl token: Bootstrapping online rl with vision-language-action models, 2026. URL https://arxiv.org/abs/2604.23073

    2026 arXiv

  32. [41]

    World action models are zero-shot policies, 2026

    Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, Ayaan Malik, Kyungmin Lee, William Liang, Nadun Ranawaka, Jiasheng Gu, Yinzhen Xu, Guanzhi Wang, Fengyuan Hu, Avnish Narayan, Johan Bjor...

    2026 arXiv

  33. [42]

    Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn

    Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware, 2023. URLhttps://arxiv.org/abs/2304.13705

    2023 arXiv

  34. [43]

    Libero-pro: Towards robust and fair evaluation of vision-language-action models beyond memorization, 2026

    Xueyang Zhou, Yangming Xu, Guiyao Tie, Yongchao Chen, Guowen Zhang, Duanfeng Chu, Pan Zhou, and Lichao Sun. Libero-pro: Towards robust and fair evaluation of vision-language-action models beyond memorization, 2026. URLhttps://arxiv.org/abs/2510.03827

    2026 arXiv

  35. [44]

    Efficient online reinforce- ment learning fine-tuning need not retain offline data, 2025

    Zhiyuan Zhou, Andy Peng, Qiyang Li, Sergey Levine, and Aviral Kumar. Efficient online reinforce- ment learning fine-tuning need not retain offline data, 2025. URLhttps://arxiv.org/abs/ 2412.07762. 19 Adaptation of Generalist Robot Policies with Minimal Data Appendices A. Actio...

    2025 arXiv

  36. [49]

    Put both the alphabet soup and the tomato sauce in the basket

  37. [50]

    Put both the cream cheese box and the butter in the basket

  38. [51]

    Turn on the stove and put the moka pot on it

  39. [52]

    Put the black bowl in the bottom drawer of the cabinet and close it

  40. [53]

    Put the white mug on the left plate and put the yellow and white mug on the right plate

  41. [54]

    Pick up the book and place it in the back compartment of the caddy

  42. [55]

    Put the white mug on the plate and put the chocolate pudding to the right of the plate

  43. [56]

    Put both the alphabet soup and the cream cheese box in the basket

  44. [57]

    Put both moka pots on the stove

  45. [58]

    The environment provides two camera views: anagentview third-person camera and arobot0_eye_in_hand wrist camera, both rendered at224×224 pixels

    Put the yellow and white mug in the microwave and close it Each task has 50 human-teleoperated demonstrations available in the benchmark; we use only𝐾= 1 for adaptation in our main experiments. The environment provides two camera views: anagentview third-person camera and arob...

  46. [59]

    Banana: Fridge Drawer→ Shelf( PickPlaceFridgeDrawerToShelf): Pick a banana from a fridge drawer and place it on a shelf

  47. [60]

    HotDog: Counter→Cabinet( PickPlaceCounterToCabinet): Pick a hot dog from the counter and place it in a cabinet

  48. [61]

    Mug→ Coffee Machine + Start(PrepareCoffee): Place a mug under the coffee machine and press the start button

  49. [62]

    put both moka pots on the stove

    Cup+Bowl →Dishwasher+Close( LoadDishwasher): Loadacupandbowlintothedishwasher and close it. 21 Adaptation of Generalist Robot Policies with Minimal Data Figure 9:Libero-Long Learning Curves forMiDASwith baseline comparison. Solid line shows mean across 3 seeds and shaded regio...

  50. [2021]

    URLhttps://arxiv.org/abs/2106.06860

    arXiv

  51. [2022]

    URLhttps://arxiv.org/abs/2205.05638

    arXiv

  52. [2024]

    URLhttps://arxiv.org/abs/2303.05479

    arXiv

  53. [2025]

    URLhttps://arxiv.org/abs/2411.12633

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.