REVIEW 4 cited by
Real2Code: Reconstruct Articulated Objects via Code Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present Real2Code, a novel approach to reconstructing articulated objects via code generation. Given visual observations of an object, we first reconstruct its part geometry using an image segmentation model and a shape completion model. We then represent the object parts with oriented bounding boxes, which are input to a fine-tuned large language model (LLM) to predict joint articulation as code. By leveraging pre-trained vision and language models, our approach scales elegantly with the number of articulated parts, and generalizes from synthetic training data to real world objects in unstructured environments. Experimental results demonstrate that Real2Code significantly outperforms previous state-of-the-art in reconstruction accuracy, and is the first approach to extrapolate beyond objects' structural complexity in the training set, and reconstructs objects with up to 10 articulated parts. When incorporated with a stereo reconstruction model, Real2Code also generalizes to real world objects from a handful of multi-view RGB images, without the need for depth or camera information.
Forward citations
Cited by 4 Pith papers
-
Fail2Progress: Learning from Real-World Robot Failures with Stein Variational Inference
Fail2Progress generates failure-targeted simulation data via Stein variational inference and fine-tunes skill effect models, improving long-horizon manipulation success rates and generalizing to unseen object counts a...
-
SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting
SplArt estimates revolute or prismatic joint parameters and part-level 3D Gaussian geometry from two sets of posed RGB images using self-supervised multi-stage optimization.
-
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards
IKER uses VLM-generated keypoint rewards to train manipulation policies in simulation that transfer to a real robot, enabling multi-step tasks and replanning.
-
SPLATART: Articulated Gaussian Splatting with Estimated Object Structure
SPLATART builds Gaussian splat renderers that can redraw articulated objects in new configurations, using sparse part segmentations and estimating joint structure for deep kinematic chains.
Discussion (0). Continue with ORCID to comment.