REVIEW 4 cited by
Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements in multimodal large language models (LLMs) have demonstrated significant potential across various domains, particularly in concept reasoning. However, their applications in understanding 3D environments remain limited, primarily offering textual or numerical outputs without generating dense, informative segmentation masks. This paper introduces Reason3D, a novel LLM designed for comprehensive 3D understanding. Reason3D processes point cloud data and text prompts to produce textual responses and segmentation masks, enabling advanced tasks such as 3D reasoning segmentation, hierarchical searching, express referring, and question answering with detailed mask outputs. We propose a hierarchical mask decoder that employs a coarse-to-fine approach to segment objects within expansive scenes. It begins with a coarse location estimation, followed by object mask estimation, using two unique tokens predicted by LLMs based on the textual query. Experimental results on large-scale ScanNet and Matterport3D datasets validate the effectiveness of our Reason3D across various tasks.
Forward citations
Cited by 4 Pith papers
-
Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM
A point-cloud LMM jointly produces text answers, 3D masks, and real-world metric measurements for object- and part-level spatial queries on indoor scenes.
-
SURPRISE3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenes
A large-scale 3D spatial reasoning segmentation benchmark with human-written queries that avoid object names shows current 3D vision-language models underperform.
-
3D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformer
A multi-purpose Omni Superpoint Transformer lets a single 3D large multimodal model achieve state-of-the-art results on 3D question answering, dense captioning, and referring segmentation using point clouds only.
-
RG-SAN: Rule-Guided Spatial Awareness Network for End-to-End 3D Referring Expression Segmentation
A rule-guided spatial-aware network that localizes all mentioned entities in a 3D scene and uses target-position weak supervision raises ScanRefer 3D-RES mIoU from 39.5 to 44.6.
Discussion (0). Continue with ORCID to comment.