pith. sign in

arxiv: 2606.12372 · v1 · pith:MNCCDCXMnew · submitted 2026-06-10 · 💻 cs.RO · cs.LG

UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning

Haoyuan Deng , Yitong Gao , Yudong Lin , Haichao Liu , Zhenyu Wu , Ziwei Wang This is my paper

Pith reviewed 2026-06-27 09:45 UTC · model grok-4.3

classification 💻 cs.RO cs.LG
keywords human-in-the-loop reinforcement learningreal-world robotic manipulationautonomous interventionvalue-risk criticgoal-conditioned recovery policypolicy stagnation detection
0
0 comments X

The pith

UniIntervene lets a robot detect its own stagnation in reinforcement learning and recover autonomously using value estimates and past episodes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces UniIntervene to reduce the heavy human oversight required in human-in-the-loop reinforcement learning for robotic manipulation tasks. It claims that future-conditioned action-value estimation produces a stable progress signal, which a temporal value-risk critic uses to spot sustained stagnation or degradation and trigger intervention. When triggered, the system pulls a high-value recovery target from memory and executes corrections through a goal-conditioned policy, shifting most interventions away from humans. A sympathetic reader would care because frequent human corrections raise labor costs and limit how far real-world robot learning can scale. If the approach works, real-world RL training becomes less intervention-intensive while maintaining or improving task success.

Core claim

UniIntervene first performs future-conditioned action-value estimation to predict the latent consequence of the current action and evaluate its induced value. A temporal value-risk critic then aggregates recent value dynamics and triggers intervention on sustained stagnation or degradation. When intervention occurs, the system retrieves a high-value recovery target from memory of past episodes and generates corrective actions via a goal-conditioned recovery policy, turning intervention into an autonomous value-aware recovery process.

What carries the argument

The temporal value-risk critic that aggregates value dynamics to trigger autonomous recovery, backed by future-conditioned action-value estimation and a goal-conditioned recovery policy that retrieves targets from memory.

If this is right

  • Average success rate across diverse real-world manipulation tasks rises by 8.6 percent over state-of-the-art HiL-RL baselines.
  • Human interventions drop by 57 percent while the policy still reaches higher-value states.
  • Intervention shifts from passive human correction to an autonomous value-aware recovery process.
  • Real-world RL becomes more scalable because labor cost per training run falls.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same detection and recovery structure could be tested in non-manipulation RL settings if the value critic generalizes beyond robotics.
  • If the memory of past interventions grows over many tasks, recovery targets might become richer and further lower intervention needs.
  • A direct comparison of training wall-clock time, rather than just intervention count, would clarify whether autonomous recovery also shortens overall learning duration.

Load-bearing premise

Future-conditioned action-value estimation plus the temporal value-risk critic can reliably detect stagnation or degradation and trigger effective autonomous recovery without introducing instability that would require extra human overrides.

What would settle it

If experiments on additional real-world manipulation tasks show that success rates do not rise or that human intervention counts remain comparable to prior HiL-RL baselines, the central effectiveness claim would not hold.

read the original abstract

Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance. However, current HiL-RL frameworks remain intervention-intensive, relying on frequent human corrections to redirect the policy out of unproductive exploration, which incurs high labor cost and limits real-world scalability. To address this, we propose UniIntervene, an agentic intervention model that detects unproductive exploration and autonomously recovers the policy toward high-value states, taking over the bulk of interventions from human operators. Specifically, UniIntervene first performs future-conditioned action-value estimation, predicting the latent consequence of the current action and evaluating its induced value, which provides a more stable progress signal. Building on this, a temporal value-risk critic aggregates recent value dynamics and triggers intervention when the estimated value exhibits sustained stagnation or degradation. When intervention is required, UniIntervene retrieves a high-value recovery target from a memory of past intervention episodes and produces executable corrective actions through a goal-conditioned recovery policy. In this way, UniIntervene turns intervention from passive human correction into a value-aware recovery process for efficient real-world RL. Extensive experiments on diverse real-world manipulation tasks demonstrate that UniIntervene improves the average success rate by 8.6% while reducing human interventions by 57% relative to state-of-the-art HiL-RL baselines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The manuscript proposes UniIntervene, an agentic intervention framework for human-in-the-loop RL in real-world robotic manipulation. It performs future-conditioned action-value estimation to predict action consequences, uses a temporal value-risk critic to detect sustained stagnation or degradation, and triggers autonomous recovery via a goal-conditioned policy that retrieves targets from an intervention memory. The central claim is that this reduces human interventions while improving policy performance, with experiments on diverse manipulation tasks showing an 8.6% higher average success rate and 57% fewer human interventions relative to SOTA HiL-RL baselines.

Significance. If the empirical gains are shown to be robust, the work addresses a key scalability barrier in real-world HiL-RL by shifting the majority of interventions from humans to an autonomous recovery process, which could enable longer-horizon learning with lower labor costs.

major comments (2)
  1. [Abstract] Abstract: The central quantitative claims (8.6% success-rate improvement and 57% reduction in interventions) are stated without any accompanying information on the number of trials per task, statistical significance testing, variance across runs, baseline implementation details, or task-selection criteria, preventing assessment of whether the data support the claims.
  2. [Method] Method (temporal value-risk critic description): The assumption that the critic can reliably identify stagnation or degradation to trigger effective recovery without introducing instability is load-bearing for the reported reduction in human interventions, yet no analysis, false-positive rates, or ablation on recovery success is provided to substantiate it.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major comment point by point below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central quantitative claims (8.6% success-rate improvement and 57% reduction in interventions) are stated without any accompanying information on the number of trials per task, statistical significance testing, variance across runs, baseline implementation details, or task-selection criteria, preventing assessment of whether the data support the claims.

    Authors: We agree that the abstract would benefit from additional context to support the claims. In the revision we will expand the abstract to state that results are averaged over 10 independent runs per task (with standard deviations reported), that statistical significance was assessed via paired t-tests (p < 0.05), and that tasks were selected to cover diverse manipulation challenges while baselines follow their original implementations. These details already appear in Section 4; the abstract update will make them immediately visible. revision: yes

  2. Referee: [Method] Method (temporal value-risk critic description): The assumption that the critic can reliably identify stagnation or degradation to trigger effective recovery without introducing instability is load-bearing for the reported reduction in human interventions, yet no analysis, false-positive rates, or ablation on recovery success is provided to substantiate it.

    Authors: The referee correctly notes that the temporal value-risk critic is central to the intervention reduction. Although the overall empirical gains are shown, the initial submission did not include dedicated analysis of false-positive rates or an ablation isolating recovery success. We will add both in the revision: a quantitative breakdown of detection accuracy (including false positives) and an ablation comparing performance with and without the critic. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The provided abstract and description present UniIntervene as a composite architecture (future-conditioned action-value estimation feeding a temporal value-risk critic that triggers goal-conditioned recovery) built atop standard RL primitives. No equations, fitted parameters renamed as predictions, or self-citation chains appear in the text that would reduce the central claims to inputs by construction. The reported gains are framed as empirical outcomes on real-world tasks rather than derived quantities forced by the method's own definitions. This is the expected non-finding for a methods paper whose load-bearing steps remain externally falsifiable.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No mathematical details, free parameters, axioms, or invented entities are specified in the abstract; the description remains at the level of high-level algorithmic components without equations or explicit assumptions.

pith-pipeline@v0.9.1-grok · 5793 in / 1237 out tokens · 24839 ms · 2026-06-27T09:45:15.840300+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 4 linked inside Pith

  1. [1]

    Constrained policy optimization

    Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In International conference on machine learning, pages 22–31. Pmlr, 2017

    2017

  2. [2]

    Juicer: Data-efficient imitation learning for robotic assembly

    Lars Ankile, Anthony Simeonov, Idan Shenfeld, and Pulkit Agrawal. Juicer: Data-efficient imitation learning for robotic assembly. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5096–5103. IEEE, 2024

    2024

  3. [3]

    V-jepa 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985, 2025

    Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Conservative safety critics for exploration.arXiv preprint arXiv:2010.14497, 2020

    Homanga Bharadhwaj, Aviral Kumar, Nicholas Rhinehart, Sergey Levine, Florian Shkurti, and Animesh Garg. Conservative safety critics for exploration.arXiv preprint arXiv:2010.14497, 2020

    arXiv 2010

  5. [5]

    Trends and challenges in robot manipulation.Science, 364(6446):eaat8414, 2019

    Aude Billard and Danica Kragic. Trends and challenges in robot manipulation.Science, 364(6446):eaat8414, 2019

    2019

  6. [6]

    In9thAnnualConferenceonRobotLearning, 2025

    Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Manuel Y Galliker, et al.𝜋0.5: a vision- language-actionmodelwithopen-worldgeneralization. In9thAnnualConferenceonRobotLearning, 2025

    2025

  7. [7]

    KevinBlack,NoahBrown,DannyDriess,AdnanEsmail,MichaelEqui,ChelseaFinn,NiccoloFusai, Lachy Groom, Karol Hausman, Brian Ichter, et al.𝜋0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024. 9

    Pith/arXiv arXiv 2024

  8. [8]

    Rlingua: Improving reinforcement learning sample efficiency in robotic manipulations with large language models.IEEE Robotics and Automation Letters, 9(7):6075–6082, 2024

    Liangliang Chen, Yutian Lei, Shiyu Jin, Ying Zhang, and Liangjun Zhang. Rlingua: Improving reinforcement learning sample efficiency in robotic manipulations with large language models.IEEE Robotics and Automation Letters, 9(7):6075–6082, 2024

    2024

  9. [9]

    Conrft: A reinforced fine-tuning method for vla models via consistency policy.Proceedings of Robotics: Science and Systems (RSS), 2025

    Yuhui Chen, Shuai Tian, Shugao Liu, Yingting Zhou, Haoran Li, and Dongbin Zhao. Conrft: A reinforced fine-tuning method for vla models via consistency policy.Proceedings of Robotics: Science and Systems (RSS), 2025

    2025

  10. [10]

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, andShuranSong.Diffusionpolicy: Visuomotorpolicylearningviaactiondiffusion.TheInternational Journal of Robotics Research, 44(10-11):1684–1704, 2025

    2025

  11. [11]

    E2hil: Entropy-guided sample selection for efficient real-world human-in-the- loop reinforcement learning.IEEE Robotics and Automation Letters, 2026

    Haoyuan Deng, Yudong Lin, Yuanjiang Xue, Haoyang Du, Qianzhun Wang, Boyang Zhou, Zhenyu Wu, and Ziwei Wang. E2hil: Entropy-guided sample selection for efficient real-world human-in-the- loop reinforcement learning.IEEE Robotics and Automation Letters, 2026

    2026

  12. [12]

    A survey on reinforcement learning of vision-language- action models for robotic manipulation.Authorea Preprints, 2025

    Haoyuan Deng, Zhenyu Wu, Haichao Liu, Wenkai Guo, Yuquan Xue, Ziyu Shan, Chuanrui Zhang, Bofang Jia, Yuan Ling, Guanxing Lu, et al. A survey on reinforcement learning of vision-language- action models for robotic manipulation.Authorea Preprints, 2025

    2025

  13. [13]

    Leave no trace: Learning to reset for safe and autonomous reinforcement learning.arXiv preprint arXiv:1711.06782, 2017

    Benjamin Eysenbach, Shixiang Gu, Julian Ibarz, and Sergey Levine. Leave no trace: Learning to reset for safe and autonomous reinforcement learning.arXiv preprint arXiv:1711.06782, 2017

    Pith/arXiv arXiv 2017

  14. [14]

    Reset-free reinforcement learning via multi-task learning: Learning dexterous manipulation behaviors without human intervention

    AbhishekGupta,JustinYu,TonyZZhao,VikashKumar,AaronRovinsky,KelvinXu,ThomasDevlin, and Sergey Levine. Reset-free reinforcement learning via multi-task learning: Learning dexterous manipulation behaviors without human intervention. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 6664–6671. IEEE, 2021

    2021

  15. [15]

    Contrastive preference learning: Learning from human feedback without rl.URL https://arxiv

    Joey Hejna, Rafael Rafailov, Harshit Sikchi, Chelsea Finn, Scott Niekum, W Bradley Knox, and Dorsa Sadigh. Contrastive preference learning: Learning from human feedback without rl.URL https://arxiv. org/abs/2310.13639, 2024

    arXiv 2024

  16. [16]

    Thriftydagger: Budget-aware novelty and risk gating for interactive imitation learning

    Ryan Hoque, Ashwin Balakrishna, Ellen Novoseller, Albert Wilcox, Daniel S Brown, and Ken Goldberg. Thriftydagger: Budget-aware novelty and risk gating for interactive imitation learning. In Conference on Robot Learning, pages 598–608. PMLR, 2022

    2022

  17. [17]

    Imitation bootstrapped reinforcement learning

    Hengyuan Hu, Suvir Mirchandani, and Dorsa Sadigh. Imitation bootstrapped reinforcement learning. arXiv preprint arXiv:2311.02198, 2023

    arXiv 2023

  18. [18]

    Reboot: Reuse data for bootstrapping efficient real-world dexterous manipulation

    Zheyuan Hu, Aaron Rovinsky, Jianlan Luo, Vikash Kumar, Abhishek Gupta, and Sergey Levine. Reboot: Reuse data for bootstrapping efficient real-world dexterous manipulation. InConference on Robot Learning, pages 1930–1949. PMLR, 2023

    1930

  19. [19]

    Transic: Sim-to-real policy transfer by learning from online correction.Conference on Robot Learning, 2025

    Yunfan Jiang, Chen Wang, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. Transic: Sim-to-real policy transfer by learning from online correction.Conference on Robot Learning, 2025

    2025

  20. [20]

    Hg-dagger: Interactive imitation learning with human experts

    Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell, and Mykel J Kochenderfer. Hg-dagger: Interactive imitation learning with human experts. In2019 International Conference on Robotics and Automation (ICRA), pages 8077–8083. IEEE, 2019

    2019

  21. [21]

    Automating reinforcement learning with example-based resets.IEEE Robotics and Automation Letters (RAL), 7(3):6606–6613, 2022

    Jigang Kim, J Hyeon Park, Daesol Cho, and H Jin Kim. Automating reinforcement learning with example-based resets.IEEE Robotics and Automation Letters (RAL), 7(3):6606–6613, 2022

    2022

  22. [22]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, et al. Openvla: An open-source vision-language-action model. InConference on Robot Learning, pages 2679–2713. PMLR, 2025. 10

    2025

  23. [23]

    Conservative q-learning for offline reinforcement learning.Advances in neural information processing systems, 33:1179–1191, 2020

    Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning.Advances in neural information processing systems, 33:1179–1191, 2020

    2020

  24. [24]

    Failure-aware rl: Reliable offline-to-online reinforcement learning with self-recovery for real-world manipulation.arXiv preprint arXiv:2601.07821, 2026

    Huanyu Li, Kun Lei, Sheng Zang, Kaizhe Hu, Yongyuan Liang, Bo An, Xiaoli Li, and Huazhe Xu. Failure-aware rl: Reliable offline-to-online reinforcement learning with self-recovery for real-world manipulation.arXiv preprint arXiv:2601.07821, 2026

    arXiv 2026

  25. [25]

    Model-based runtime monitoring with interactive imitation learning

    Huihan Liu, Shivin Dass, Roberto Martín-Martín, and Yuke Zhu. Model-based runtime monitoring with interactive imitation learning. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 4154–4161. IEEE, 2024

    2024

  26. [26]

    Reset-free lifelong learning with skill-space planning.arXiv preprint arXiv:2012.03548, 2020

    Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Reset-free lifelong learning with skill-space planning.arXiv preprint arXiv:2012.03548, 2020

    arXiv 2012

  27. [27]

    Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning.Science Robotics, pages 1–54, 2025

    J Luo, C Xu, J Wu, and S Levine. Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning.Science Robotics, pages 1–54, 2025

    2025

  28. [28]

    Serl: A software suite for sample-efficient robotic reinforcement learning

    Jianlan Luo, Zheyuan Hu, Charles Xu, You Liang Tan, Jacob Berg, Archit Sharma, Stefan Schaal, Chelsea Finn, Abhishek Gupta, and Sergey Levine. Serl: A software suite for sample-efficient robotic reinforcement learning. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 16961–16969, 2024

    2024

  29. [29]

    Human-in-the-loop imitation learning using remote teleoperation.arXiv preprint arXiv:2012.06733, 2020

    Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, and Silvio Savarese. Human-in-the-loop imitation learning using remote teleoperation.arXiv preprint arXiv:2012.06733, 2020

    arXiv 2012

  30. [30]

    Sop: A scalable online post-training system for vision-language-action models.arXiv preprint arXiv:2601.03044, 2026

    Mingjie Pan, Siyuan Feng, Qinglin Zhang, Xinchen Li, Jianheng Song, Chendi Qu, Yi Wang, Chuankang Li, Ziyu Xiong, Zhi Chen, et al. Sop: A scalable online post-training system for vision-language-action models.arXiv preprint arXiv:2601.03044, 2026

    arXiv 2026

  31. [31]

    Fast: Efficient action tokenization for vision-language-action models, 2025

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models, 2025

    2025

  32. [32]

    Areductionofimitationlearningandstructured prediction to no-regret online learning

    StéphaneRoss,GeoffreyGordon,andDrewBagnell. Areductionofimitationlearningandstructured prediction to no-regret online learning. InProceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627–635. JMLR Workshop and Conference Proceedings, 2011

    2011

  33. [33]

    A state-distribution matching approach to non-episodic reinforcement learning

    Archit Sharma, Rehaan Ahmad, and Chelsea Finn. A state-distribution matching approach to non-episodic reinforcement learning. InInternational Conference on Machine Learning, pages 19645–19657. PMLR, 2022

    2022

  34. [34]

    Autonomous reinforcement learning via subgoal curricula.Proceedings of Advances in Neural Information Processing Systems, 34:18474–18486, 2021

    Archit Sharma, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Autonomous reinforcement learning via subgoal curricula.Proceedings of Advances in Neural Information Processing Systems, 34:18474–18486, 2021

    2021

  35. [35]

    Learning to be safe: Deep rl with a safety critic.arXiv preprint arXiv:2010.14603, 2020

    Krishnan Srinivasan, Benjamin Eysenbach, Sehoon Ha, Jie Tan, and Chelsea Finn. Learning to be safe: Deep rl with a safety critic.arXiv preprint arXiv:2010.14603, 2020

    arXiv 2010

  36. [36]

    Responsive safety in reinforcement learning by pid lagrangian methods

    Adam Stooke, Joshua Achiam, and Pieter Abbeel. Responsive safety in reinforcement learning by pid lagrangian methods. InInternational conference on machine learning, pages 9133–9143. PMLR, 2020

    2020

  37. [37]

    Ig-rft: An interaction-guided rl framework for vla models in long-horizon robotic manipulation.arXiv preprint arXiv:2602.20715, 2026

    Zhian Su, Weijie Kong, Haonan Dong, and Huixu Dong. Ig-rft: An interaction-guided rl framework for vla models in long-horizon robotic manipulation.arXiv preprint arXiv:2602.20715, 2026. 11

    arXiv 2026

  38. [38]

    Gigabrain-0.5 m*: a vla that learns from world model-based reinforcement learning.arXiv preprint arXiv:2602.12099, 2026

    GigaBrain Team, Boyuan Wang, Bohan Li, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Jie Li, Jindi Lv, Jingyu Liu, et al. Gigabrain-0.5 m*: a vla that learns from world model-based reinforcement learning.arXiv preprint arXiv:2602.12099, 2026

    arXiv 2026

  39. [39]

    Recovery rl: Safe reinforcement learning with learned recovery zones.IEEE Robotics and Automation Letters (RAL), 6(3):4915–4922, 2021

    Brijen Thananjeyan, Ashwin Balakrishna, Suraj Nair, Michael Luo, Krishnan Srinivasan, Minho Hwang, Joseph E Gonzalez, Julian Ibarz, Chelsea Finn, and Ken Goldberg. Recovery rl: Safe reinforcement learning with learned recovery zones.IEEE Robotics and Automation Letters (RAL), 6(3):4915–4922, 2021

    2021

  40. [40]

    Learning while deploying: Fleet-scale reinforcement learning for generalist robot policies.arXiv preprint arXiv:2605.00416, 2026

    Yi Wang, Xinchen Li, Pengwei Xie, Pu Yang, Buqing Nie, Yunuo Cai, Qinglin Zhang, Chendi Qu, Jeffrey Wu, Jianheng Song, et al. Learning while deploying: Fleet-scale reinforcement learning for generalist robot policies.arXiv preprint arXiv:2605.00416, 2026

    Pith/arXiv arXiv 2026

  41. [41]

    When to ask for help: Proactive interventionsinautonomousreinforcementlearning.ProceedingsofAdvancesinNeuralInformation Processing Systems, 35:16918–16930, 2022

    Annie Xie, Fahim Tajwar, Archit Sharma, and Chelsea Finn. When to ask for help: Proactive interventionsinautonomousreinforcementlearning.ProceedingsofAdvancesinNeuralInformation Processing Systems, 35:16918–16930, 2022

    2022

  42. [42]

    Continual learning of control primitives: Skill discovery via reset-games.Advances in Neural Information Processing Systems, 33:4999–5010, 2020

    Kelvin Xu, Siddharth Verma, Chelsea Finn, and Sergey Levine. Continual learning of control primitives: Skill discovery via reset-games.Advances in Neural Information Processing Systems, 33:4999–5010, 2020

    2020

  43. [43]

    Compliant residual dagger: Improving real-world contact-rich manipulation with human corrections.arXiv preprint arXiv:2506.16685, 2025

    Xiaomeng Xu, Yifan Hou, Zeyi Liu, and Shuran Song. Compliant residual dagger: Improving real-world contact-rich manipulation with human corrections.arXiv preprint arXiv:2506.16685, 2025

    arXiv 2025

  44. [44]

    Armada: Autonomous online failure detection and human shared control empower scalable real-world deployment and adaptation

    Wenye Yu, Jun Lv, Zixi Ying, Yang Jin, Chuan Wen, and Cewu Lu. Armada: Autonomous online failure detection and human shared control empower scalable real-world deployment and adaptation. arXiv preprint arXiv:2510.02298, 2025

    arXiv 2025

  45. [45]

    Safevla: Towards safety alignment of vision-language-action model via constrained learning

    Borong Zhang, Yuhao Zhang, Jiaming Ji, Yingshan Lei, Juntao Dai, Yuanpei Chen, and Yaodong Yang. Safevla: Towards safety alignment of vision-language-action model via constrained learning. Advances in Neural Information Processing Systems, 38:153335–153373, 2026

    2026

  46. [46]

    Reinforcement learning for robot research: A comprehensive review and open issues.International Journal of Advanced Robotic Systems, 2021

    Tengteng Zhang and Hongwei Mo. Reinforcement learning for robot research: A comprehensive review and open issues.International Journal of Advanced Robotic Systems, 2021

    2021

  47. [47]

    Empowering embodied manipulation: A bimanual-mobile robot manipulation dataset for household tasks.arXiv preprint arXiv:2405.18860, pages 1–24, 2024

    Tianle Zhang, Dongjiang Li, Yihang Li, Zecui Zeng, Lin Zhao, Lei Sun, Yue Chen, Xuelong Wei, Yibing Zhan, Lusong Li, et al. Empowering embodied manipulation: A bimanual-mobile robot manipulation dataset for household tasks.arXiv preprint arXiv:2405.18860, pages 1–24, 2024

    arXiv 2024

  48. [48]

    Grape: Generalizing robot policy via preference alignment

    Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, Siwei Han, Chaoqi Wang, Mingyu Ding, Dieter Fox, and Huaxiu Yao. Grape: Generalizing robot policy via preference alignment. arXiv preprint arXiv:2411.19309, 2024

    arXiv 2024

  49. [49]

    The ingredients of real-world robotic reinforcement learning.arXiv preprint arXiv:2004.12570, 2020

    HenryZhu,JustinYu,AbhishekGupta,DhruvShah,KristianHartikainen,AviSingh,VikashKumar, and Sergey Levine. The ingredients of real-world robotic reinforcement learning.arXiv preprint arXiv:2004.12570, 2020. 12 Appendix A Overview This supplement details the full method behindUniIntervene, from the proxy value function used to score rollouts, through the tempo...

    arXiv 2004