REVIEW 7 cited by
OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The development of general robotic systems capable of manipulating in unstructured environments is a significant challenge. While Vision-Language Models(VLM) excel in high-level commonsense reasoning, they lack the fine-grained 3D spatial understanding required for precise manipulation tasks. Fine-tuning VLM on robotic datasets to create Vision-Language-Action Models(VLA) is a potential solution, but it is hindered by high data collection costs and generalization issues. To address these challenges, we propose a novel object-centric representation that bridges the gap between VLM's high-level reasoning and the low-level precision required for manipulation. Our key insight is that an object's canonical space, defined by its functional affordances, provides a structured and semantically meaningful way to describe interaction primitives, such as points and directions. These primitives act as a bridge, translating VLM's commonsense reasoning into actionable 3D spatial constraints. In this context, we introduce a dual closed-loop, open-vocabulary robotic manipulation system: one loop for high-level planning through primitive resampling, interaction rendering and VLM checking, and another for low-level execution via 6D pose tracking. This design ensures robust, real-time control without requiring VLM fine-tuning. Extensive experiments demonstrate strong zero-shot generalization across diverse robotic manipulation tasks, highlighting the potential of this approach for automating large-scale simulation data generation.
Forward citations
Cited by 7 Pith papers
-
RoboReact: Agentic Skill Distillation from Generated Egocentric Videos for Generalizable Whole-Body Manipulation
From a single egocentric RGB-D frame and a language instruction, RoboReact generates a human manipulation video, distills it into object-relative keyframe skills, refines them through a vision-language-model trial loo...
-
A Closed-Loop Multi-Agent Framework for Robust Multi-Robot Manipulation
A closed-loop multi-agent LLM framework enables heterogeneous robots to collaboratively manipulate objects by decomposing tasks, grounding actions via visual tools, and recovering from execution failures hierarchically.
-
FineGrasp: Towards Robust Grasping for Delicate Objects
FineGrasp improves 6-DoF grasp detection for small/delicate objects using instance-normalized graspness labels, multi-range attention, surface-normal priors, and sim-to-real training.
-
T-Rex: Task-Adaptive Spatial Representation Extraction for Robotic Manipulation with Vision-Language Models
A zero-training framework that adaptively selects spatial representation extractors per object and per task stage improves real-world robot manipulation success and efficiency over fixed-representation baselines.
-
CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulation
A first benchmark that tests whether robots can read multi-page appliance manuals and then plan and execute manipulation tasks on appliances in simulation.
-
Object-Focus Actor for Data-efficient Robot Generalization Dexterous Manipulation
Object-Focus Actor makes dexterous manipulation policies generalize to new object positions and backgrounds by focusing on the hand-object region and using relative poses and actions, needing only 10 to 30 demonstrations.
-
3DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Model
A diffusion world model predicts 3D optical flow as an embodiment-agnostic action plan, and constrained optimization converts the flow into robot arm actions.
Discussion (0). Continue with ORCID to comment.