REVIEW 6 cited by
Gaussian Grouping: Segment and Edit Anything in 3D Scenes
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The recent Gaussian Splatting achieves high-quality and real-time novel-view synthesis of the 3D scenes. However, it is solely concentrated on the appearance and geometry modeling, while lacking in fine-grained object-level scene understanding. To address this issue, we propose Gaussian Grouping, which extends Gaussian Splatting to jointly reconstruct and segment anything in open-world 3D scenes. We augment each Gaussian with a compact Identity Encoding, allowing the Gaussians to be grouped according to their object instance or stuff membership in the 3D scene. Instead of resorting to expensive 3D labels, we supervise the Identity Encodings during the differentiable rendering by leveraging the 2D mask predictions by Segment Anything Model (SAM), along with introduced 3D spatial consistency regularization. Compared to the implicit NeRF representation, we show that the discrete and grouped 3D Gaussians can reconstruct, segment and edit anything in 3D with high visual quality, fine granularity and efficiency. Based on Gaussian Grouping, we further propose a local Gaussian Editing scheme, which shows efficacy in versatile scene editing applications, including 3D object removal, inpainting, colorization, style transfer and scene recomposition. Our code and models are at https://github.com/lkeab/gaussian-grouping.
Forward citations
Cited by 6 Pith papers
-
Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generation
Hallo4D uses vision-language models to detect and correct spatial and temporal mistakes in AI-generated 3D and 4D content, improving consistency without retraining the base generators.
-
LabelGS: Label-Aware 3D Gaussian Splatting for 3D Scene Segmentation
LabelGS assigns 2D video-tracking labels to the most-contributing 3D Gaussians, with depth-based occlusion masking and a projection filter, reporting better mIoU/PSNR than Feature-3DGS with roughly 22x faster training.
-
DCHM: Depth-Consistent Human Modeling for Multiview Detection
DCHM uses superpixel-based Gaussian Splatting to make monocular depth estimates multiview-consistent, producing point clouds that yield state-of-the-art label-free pedestrian detection on Wildtrack, Terrace, and MultiviewX.
-
MUVOD: A Novel Multi-view Video Object Segmentation Dataset and A Benchmark for 3D Segmentation
MUVOD provides multi-view video panoptic masks for 17 dynamic scenes (459 instances, 73 categories) and benchmarks for multi-view video and 3D object segmentation.
-
DSG-World: Learning a 3D Gaussian World Model from Dual State Videos
DSG-World builds two segmented 3D Gaussian fields from two scene states and trains them with mutual consistency, enabling novel-state simulation without inpainting or dense capture.
-
Hi-LSplat: Hierarchical 3D Language Gaussian Splatting
Hi-LSplat trains language-augmented 3D Gaussians with a three-level semantic tree and instance/part contrastive losses, improving open-vocabulary 3D segmentation and localization on eight datasets.
Discussion (0). Continue with ORCID to comment.