REVIEW 5 cited by
Interactive Robot Learning from Verbal Correction
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The ability to learn and refine behavior after deployment has become ever more important for robots as we design them to operate in unstructured environments like households. In this work, we design a new learning system based on large language model (LLM), OLAF, that allows everyday users to teach a robot using verbal corrections when the robot makes mistakes, e.g., by saying "Stop what you're doing. You should move closer to the cup." A key feature of OLAF is its ability to update the robot's visuomotor neural policy based on the verbal feedback to avoid repeating mistakes in the future. This is in contrast to existing LLM-based robotic systems, which only follow verbal commands or corrections but not learn from them. We demonstrate the efficacy of our design in experiments where a user teaches a robot to perform long-horizon manipulation tasks both in simulation and on physical hardware, achieving on average 20.0% improvement in policy success rate. Videos and more results are at https://ut-austin-rpl.github.io/olaf/
Forward citations
Cited by 5 Pith papers
-
Effects of Robot Competency and Motion Legibility on Human Correction Feedback
A user study shows that robot competency and motion legibility shift when and whether people correct a robot, contradicting two common assumptions in learning-from-corrections.
-
Robix: A Unified Model for Robot Interaction, Reasoning and Planning
A three-stage-trained VLM unifies robot planning and dialogue, and beats commercial VLMs on the authors' interactive-task benchmarks.
-
Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework
A conceptual framework classifies human feedback to RL agents along nine dimensions and seven quality criteria, unifying human-centered, interface-centered, and model-centered design perspectives.
-
Ask-to-Clarify: Resolving Instruction Ambiguity through Multi-turn Dialogue
Ask-to-Clarify combines a dialogue VLM with a frozen diffusion policy so a robot can disambiguate instructions and then execute low-level actions on 8 real-world tasks.
-
Scene Graph-Guided Proactive Replanning for Failure-Resilient Embodied Agent
A robot replanner that compares scene graphs to successful demonstrations before each subtask, triggering LLM-based replanning on mismatch, raises task success in AI2-THOR.
Discussion (0). Continue with ORCID to comment.