REVIEW 5 cited by
Imitating Task and Motion Planning with Visuomotor Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Imitation learning is a powerful tool for training robot manipulation policies, allowing them to learn from expert demonstrations without manual programming or trial-and-error. However, common methods of data collection, such as human supervision, scale poorly, as they are time-consuming and labor-intensive. In contrast, Task and Motion Planning (TAMP) can autonomously generate large-scale datasets of diverse demonstrations. In this work, we show that the combination of large-scale datasets generated by TAMP supervisors and flexible Transformer models to fit them is a powerful paradigm for robot manipulation. To that end, we present a novel imitation learning system called OPTIMUS that trains large-scale visuomotor Transformer policies by imitating a TAMP agent. OPTIMUS introduces a pipeline for generating TAMP data that is specifically curated for imitation learning and can be used to train performant transformer-based policies. In this paper, we present a thorough study of the design decisions required to imitate TAMP and demonstrate that OPTIMUS can solve a wide variety of challenging vision-based manipulation tasks with over 70 different objects, ranging from long-horizon pick-and-place tasks, to shelf and articulated object manipulation, achieving 70 to 80% success rates. Video results and code at https://mihdalal.github.io/optimus/
Forward citations
Cited by 5 Pith papers
-
Combined Constrained Sampling and Reinforcement Learning for Robotic Manipulation
Guiding goal-conditioned reinforcement learning with samples from a constrained feasible-state manifold lets a simulated double-sphere and a Panda-arm policy succeed far more often than RL with random resets.
-
Articulate AnyMesh: Open-Vocabulary 3D Articulated Objects Modeling
Articulate AnyMesh converts arbitrary rigid 3D meshes into articulated objects by combining VLM-driven part segmentation, geometry-aware joint estimation, and optional shape completion.
-
RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills
RobotSmith autonomously designs, 3D-prints, and uses task-specific tools for robotic manipulation, raising task success from 2.8% (no tool) to 50% in simulation.
-
ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning
ReinforceGen uses imitation learning, RL fine-tuning of skill policies, and real-time pose replanning to reach over 80% success on five long-horizon Robosuite tasks from only 10 human demonstrations.
-
Steering Robots with Inference-Time Interactions
Frozen imitation policies can be steered at inference time via user interactions, with a diffusion-sampling method and a constraint-enforcing framework that provides formal task guarantees.
Discussion (0). Continue with ORCID to comment.