REVIEW 7 cited by
SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this paper, we propose a new framework for zero-shot object navigation. Existing zero-shot object navigation methods prompt LLM with the text of spatially closed objects, which lacks enough scene context for in-depth reasoning. To better preserve the information of environment and fully exploit the reasoning ability of LLM, we propose to represent the observed scene with 3D scene graph. The scene graph encodes the relationships between objects, groups and rooms with a LLM-friendly structure, for which we design a hierarchical chain-of-thought prompt to help LLM reason the goal location according to scene context by traversing the nodes and edges. Moreover, benefit from the scene graph representation, we further design a re-perception mechanism to empower the object navigation framework with the ability to correct perception error. We conduct extensive experiments on MP3D, HM3D and RoboTHOR environments, where SG-Nav surpasses previous state-of-the-art zero-shot methods by more than 10% SR on all benchmarks, while the decision process is explainable. To the best of our knowledge, SG-Nav is the first zero-shot method that achieves even higher performance than supervised object navigation methods on the challenging MP3D benchmark.
Forward citations
Cited by 7 Pith papers
-
VTM-Nav: Harnessing Cross-Episode Experience for Object-Goal Navigation with Hierarchical Visual-Topological Memory
A hierarchical room-and-object memory that persists across independent ObjectNav episodes yields small success-rate gains, but most of the gain comes from within-episode memory rather than the cross-episode component.
-
Imaginative World Modeling with Scene Graphs for Embodied Agent Navigation
SGImagineNav uses an imagined hierarchical scene graph, filled in by an LLM, that guides a robot to unseen objects and achieves 65.4% and 66.8% success on HM3D and HSSD.
-
Recursive Visual Imagination and Adaptive Linguistic Grounding for Vision Language Navigation
A VLN agent that recursively imagines future views and layouts in a fixed-size neural grid, and adaptively aligns instruction parts to grid cells, achieves state-of-the-art success rates on R2R-CE and ObjectNav.
-
SG-CoT: An Ambiguity-Aware Robotic Planning Framework using Scene Graph Representations
SG-CoT grounds an LLM planner's chain-of-thought in a scene graph via iterative API queries, improving ambiguity detection and clarification in simulated manipulation, though its success metric credits any clarifying ...
-
IRS: Instance-Level 3D Scene Graphs via Room Prior Guided LiDAR-Camera Fusion
IRS builds instance-level 3D scene graphs faster by using LiDAR room priors to constrain and parallelize semantic fusion from vision-language models.
-
MoTo: A Zero-shot Plug-in Interaction-aware Navigation for General Mobile Manipulation
MoTo turns existing fixed-base manipulation models into mobile manipulators by using VLM-picked contact keypoints and trajectory optimization to find docking points, with no training of MoTo itself.
-
From Data to Modeling: Fully Open-vocabulary Scene Graph Generation
OvSGTR jointly predicts unseen objects and relationships in scene graphs using a DETR-like transformer, relation-aware pre-training, and knowledge distillation, achieving state-of-the-art results on VG150 and GQA200.
Discussion (0). Continue with ORCID to comment.