REVIEW 5 cited by
RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Rapid progress in high-level task planning and code generation for open-world robot manipulation has been witnessed in Embodied AI. However, previous studies put much effort into general common sense reasoning and task planning capabilities of large-scale language or multi-modal models, relatively little effort on ensuring the deployability of generated code on real robots, and other fundamental components of autonomous robot systems including robot perception, motion planning, and control. To bridge this ``ideal-to-real'' gap, this paper presents \textbf{RobotScript}, a platform for 1) a deployable robot manipulation pipeline powered by code generation; and 2) a code generation benchmark for robot manipulation tasks in free-form natural language. The RobotScript platform addresses this gap by emphasizing the unified interface with both simulation and real robots, based on abstraction from the Robot Operating System (ROS), ensuring syntax compliance and simulation validation with Gazebo. We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms, and multiple grippers. Additionally, our benchmark assesses reasoning abilities for physical space and constraints, highlighting the differences between GPT-3.5, GPT-4, and Gemini in handling complex physical interactions. Finally, we present a thorough evaluation on the whole system, exploring how each module in the pipeline: code generation, perception, motion planning, and even object geometric properties, impact the overall performance of the system.
Forward citations
Cited by 5 Pith papers
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills
RobotSmith autonomously designs, 3D-prints, and uses task-specific tools for robotic manipulation, raising task success from 2.8% (no tool) to 50% in simulation.
-
ETA: A New Agentic Paradigm for Embodied Tasks
A general-purpose LLM planner using only observe, mark_point, and move_to solves 90% of 130 LIBERO manipulation tasks when allowed five attempts per task, with no robot-policy training.
-
STEP Planner: Constructing cross-hierarchical subgoal tree as an embodied long-horizon task planner
STEP builds a coarse-to-fine subgoal tree with LLM-based decomposition and termination checks, reporting higher task success than existing LLM planners on WAH-NL and a real robot.
-
Foundation Model Driven Robotics: A Comprehensive Review
A review of foundation-model-driven robotics that synthesizes recent work across perception, planning, control, HRI, simulation, and sim-to-real transfer, and highlights open challenges.
Discussion (0). Continue with ORCID to comment.