REVIEW 10 cited by
Human-in-the-Loop Imitation Learning using Remote Teleoperation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Imitation Learning is a promising paradigm for learning complex robot manipulation skills by reproducing behavior from human demonstrations. However, manipulation tasks often contain bottleneck regions that require a sequence of precise actions to make meaningful progress, such as a robot inserting a pod into a coffee machine to make coffee. Trained policies can fail in these regions because small deviations in actions can lead the policy into states not covered by the demonstrations. Intervention-based policy learning is an alternative that can address this issue -- it allows human operators to monitor trained policies and take over control when they encounter failures. In this paper, we build a data collection system tailored to 6-DoF manipulation settings, that enables remote human operators to monitor and intervene on trained policies. We develop a simple and effective algorithm to train the policy iteratively on new data collected by the system that encourages the policy to learn how to traverse bottlenecks through the interventions. We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators, and further show that our method outperforms multiple state-of-the-art baselines for learning from the human interventions on a challenging robot threading task and a coffee making task. Additional results and videos at https://sites.google.com/stanford.edu/iwr .
Forward citations
Cited by 10 Pith papers
-
Towards Human-level Dexterous Teleoperation
A single-stage RL co-tracking controller trained on consecutive human-derived hand–object subgoals achieves ~75% real-robot success on long-horizon dexterous teleoperation where baselines fail.
-
Update-Free On-Policy Steering via Verifiers
Lightweight verifiers trained on a diffusion policy’s own evaluation rollouts raise real-robot success rates ~49% on average via Best-of-N or classifier guidance, without changing base parameters.
-
Latent Policy Barrier: Learning Robust Visuomotor Policies by Staying In-Distribution
Latent Policy Barrier improves behavior-cloned visuomotor policies by using a latent dynamics model trained on expert and rollout data to guide actions back toward in-distribution expert states.
-
Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism
A learned proxy Q-function that triggers expert help when agent and expert actions diverge reduces human takeover cost and improves imitation learning efficiency in simulated driving and navigation.
-
Learning from Active Human Involvement through Proxy Value Propagation
A reward-free human-in-the-loop RL method that labels human demonstrations with high Q values and intervened agent actions with low Q values, then propagates these values through TD learning to train policies across d...
-
RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correction
Robot policies trained on human interventions that rewind to a familiar state and then correct the mistake achieve higher long-horizon success and better data efficiency than imitation on full demonstrations alone.
-
Confidence-Guided Human-AI Collaboration: Reinforcement Learning with Distributional Proxy Value Propagation for Autonomous Driving
C-HAC combines human demonstrations and reward-based RL for driving, using distributional return estimates to decide when the agent should follow the human-guided policy versus its self-learned policy.
-
Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Space
Breaking long manipulation tasks into atomic subtasks and collecting demonstrations from varied starting poses improves imitation learning success using fewer demonstration frames.
-
Imitation Learning Based on Disentangled Representation Learning of Behavioral Characteristics
A weakly-supervised CVAE with action chunking lets a robot change wiping speed online from instruction labels, but the same mechanism fails to disentangle wiping force and fails on spatial pick-and-place directives.
-
Data Pyramid for Embodied Manipulation
Embodied training data form a five-layer pyramid—real-robot, UMI, ego/exo, simulation, general V–L—ordered by the trade-off between scale and robot alignment, and model capabilities track how those layers are mixed.
Discussion (0). Continue with ORCID to comment.