Pith. sign in

REVIEW 4 major objections 5 minor 62 references

LEGO: Leveled Language Gaussian Splatting

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LEGO re-grades volatile multi-view SAM masks into a unified 3D-consistent semantic hierarchy whose levels are strictly invariant to viewing distance and absolute physical size, and uses that hierarchy for open-vocabulary segmentation and…

desk verdict Genuinely new multi-view mask re-grading mechanism and SOTA numbers on standard benchmarks, but the 'strictly invariant to absolute physical size' claim is not supported by the method, and the hierarchy itself is only evaluated on a self-created benchmark. read the letter →

arxiv 2608.10057 v1 pith:4FEBJ6WS submitted 2026-08-10 cs.CV

Yuning Peng , Haiping Wang , Yuan Liu , Yipeng Lu , Zhen Dong , Bisheng Yang This is my paper · ORCID
classification cs.CV
keywords 3DsceneunderstandingsemantichierarchyGaussiansplattingopen-vocabularysegmentationSAMmasksgraphlargelanguagemodelsspatialreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LEGO claims that the messy, view-dependent granularities produced by 2D segmentation models can be stitched into a single, 3D-consistent semantic hierarchy, one in which the same physical entity always receives the same structural level no matter how close or far the camera is or how large or small the object is in absolute terms. The key step is to lift multi-view masks into 3D, estimate each mask's physical span, and then let the local histogram of those spans reveal the latent levels, assigning every mask to the nearest histogram peak. Once levels are fixed, the method trains a Gaussian splat field with level-wise decoupled identity features, yielding hierarchical segments that can be grounded with CLIP and organized into a scene graph for LLM-driven spatial queries. If this works, automatic 3D scene understanding would move from flat object recognition to scale-free part-whole reasoning, which matters for robots and assistants that must parse cluttered scenes at multiple granularities.

What carries the argument

The load-bearing mechanism is the local peak-based mask level assignment: for each lifted SAM mask, its 3D spatial diameter $s_i = 2\sqrt{\sum_{d\in\{x,y,z\}} \mathrm{std}(\mathcal{P}_{m_i,d})^2}$ is compared with the histogram of scales of co-visible neighboring masks, and the detected peaks define the levels that place the mask via Eq. (1). This converts continuous physical scale into a discrete, locally normalized structural rank. The second mechanism is the level-wise decoupled feature field: each Gaussian carries $L$ independent $d$-dimensional features, one per level, trained by the contrastive indicator loss, which prevents cross-level interference and enables recursive HDBSCAN clustering into a nested segmentation tree.

What would settle it

Build or render a scene containing two instances of the same semantic object at strongly different physical sizes, such as a large flower and a small flower in one bouquet. If LEGO assigns them to different levels because their scales fall near different histogram peaks, then the claimed invariance to absolute physical size fails; the assignment is directly observable by inspecting the per-mask level labels computed by Eq. (1) and the resulting 3D segmentation.

Watch

Extended reading notes

Core claim

The paper argues that the three fixed 2D granularities produced by SAM are not a defect but fragmented glimpses of a deeper latent hierarchy: a close-up view of a bouquet may see "bud → petal", while a distant view sees "flowerpot → bouquet → bud", and together these observations reconstruct the full lineage. LEGO formalizes the reconstruction by computing a 3D physical scale for each lifted mask and running peak detection on the scale histogram of co-visible masks; the peaks are taken as the structural levels and Eq. (1) assigns each mask to the nearest peak. The paper claims this "3D level-based" assignment is strictly invariant to viewing distance and absolute physical size, and that supervising a Gaussian field with a dense level-wise indicator, built on monotonicity, recursive inclusion, and structural inheritance, yields a decoupled feature space in which each level can be segmented and grounded without cross-level semantic entanglement.

Load-bearing premise

The method assumes that an object's 3D physical span, measured by the spread of its lifted mask points, is a reliable proxy for its semantic structural rank, so that histogram peaks over co-visible masks recover the true level hierarchy regardless of intra-class size variation.

Editorial extensions

If this is right

  • SAM's fixed three granularities can be replaced by an arbitrary-depth hierarchy learned from the scene itself, because the level count comes from histogram peaks rather than from SAM's architecture.
  • Objects that share a structural rank, such as all buds in a bouquet or all petals on a flower, remain at the same level regardless of camera distance or physical size, which reduces the granularity blurring seen in view-dependent distillation.
  • Level-wise decoupled features give cleaner boundaries for fine-grained parts, such as corn and onion segments in the ramen scene, than flat or entangled feature fields, as reported in the open-vocabulary benchmarks.
  • A level-wise language scene graph with hierarchical and adjacency edges allows an LLM to answer compositional queries through relation-constrained beam search, a capability that flat CLIP matching lacks.
  • Promptable segmentation improves on saturated benchmarks, with up to +1.6 mIoU on NVOS and strong mIoU on SPIn-NeRF, suggesting the hierarchy also helps under manual point or scribble prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: if the level-consistency claim holds, the same machinery could be used to build a canonical part-whole taxonomy across a dataset of scenes, because levels are assigned by local histogram peaks rather than a global scale threshold, so object instances could be compared level-by-level without a hand-built ontology.
  • Inference: a stress test follows directly from the method's own initialization: since level assignment starts from physical scale, the invariance to size is only as strong as the histogram peaks' ability to separate sizes within a local neighborhood; constructing a bouquet with one flower an order of magnitude larger than its neighbors would test whether both flowers still receive the same level.
  • Inference: the scene-graph beam search is a natural scaffold for distance- or orientation-based queries; extending the graph edges with metric spatial relations would let the same retrieval chain answer "left of" or "farther than" without retraining the field.
  • Inference: because the pipeline inherits SAM's mask quality and CLIP's text-image alignment, its open-vocabulary ceiling is tied to those backbones, so swapping in stronger foundation models should improve grounding directly, suggesting the hierarchy itself is backbone-agnostic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LEGO, a method for open-vocabulary and hierarchical understanding of 3D Gaussian splatting scenes. It lifts multi-view SAM masks to 3D, estimates their physical scale, and assigns each mask a discrete level by nearest histogram peak of co-visible mask scales (Eq. 1). The level labels are used to build pixel-pair indicators with monotonicity, recursive inclusion, and structural inheritance axioms; these supervise a decoupled per-level Gaussian identity feature space. Subsequent hierarchical segmentation uses recursive HDBSCAN over the level features, CLIP grounding uses an optimal view selection, and a level-wise scene graph supports LLM-driven Chain-of-Retrieval. The paper reports state-of-the-art results on NVOS, SPIn-NeRF, LERF-OVS, Mip-NeRF 360, and a new CoR benchmark.

Significance. If validated, the method offers a practical recipe for automatic, hierarchical, part-level 3D scene decomposition from 2D foundation-model masks, and the downstream LLM-grounded graph reasoning is a promising direction. The paper is well-executed on the engineering side: the supplementary formalizes the indicator construction and losses, ablations support the individual loss components, and the qualitative CoR results are compelling. However, the central conceptual claim, that the method discovers levels that are strictly invariant to viewing distance and absolute physical size, is not established by the current formulation, since levels are inferred from physical scale histograms. This weakens the hierarchy-discovery claim and the interpretation of the benchmark improvements.

major comments (4)
  1. [Sec. 3.1, Eq. (1); Sec. 1, "Our Insight"] The claimed strict invariance of the assigned levels to absolute physical size is not supported by the method. In Eq. (1), the level index is assigned by the nearest peak of a histogram of physical scales of co-visible masks, so two same-rank entities with well-separated physical diameters, such as a large flower and a small flower in the motivating bouquet example, will form separate histogram modes and receive different level indices. No constraint, theorem, or experiment in the paper enforces that same-rank instances across scale gaps are merged into one level, and the manuscript itself in Sec. 1 calls physical scales an "initial proxy," which is in tension with the "strictly invariant" phrasing used in the same section. Because the level labels propagate into the supervision in Sec. 3.1 and the feature learning in Sec. 3.2, this is a load-bearing gap; the authors should either prove the invariance or directly evaluate it, for example by measuring level consistency across instances of known semantic rank with varied physical sizes.
  2. [Sec. 3.1, "Local Peak-based Mask Level Assignment"] The level index is defined locally, not globally. Because peaks are detected for each target mask from its co-visible neighbor set N(mi), the resulting li is a local rank within a region, and the same level index can denote different semantic ranks in different scene regions. This is in tension with the paper's abstract and with the evaluation protocol in Supp. Sec. B, which uses "the first three levels" as if levels were globally comparable. Without a definition of global level semantics or an empirical demonstration that local peak ranks coincide across regions, the claimed unified hierarchy is under-specified, and the reported open-vocabulary results may reflect local ranking artifacts rather than a scene-wide semantic hierarchy.
  3. [Sec. 3.1 and Supp. Sec. B] The hierarchy depth L is fixed to 8 and the method is expected to identify L prominent peaks, but the paper does not specify behavior when the local scale histogram has fewer than eight modes or when peak detection is ambiguous. Since Eq. (1) depends critically on the peak set, the free choice of L and the peak-detection parameters should be analyzed with a sensitivity study. As written, the adaptivity claim is not fully supported, and the reader cannot determine whether the reported gains are robust to these choices.
  4. [Supp. Sec. B, "Open-Vocabulary Understanding"] The open-vocabulary evaluation protocol retrieves all clusters within the selected level whose similarity exceeds 0.9*Smax, but because the selected level is the local level of the best-matching cluster, there is no guarantee that the retrieved clusters are semantically co-ranked with the target. This protocol, combined with the local level assignment of Sec. 3.1, makes the reported gains on LERF-OVS and Mip-NeRF 360 difficult to interpret as evidence for a unified semantic hierarchy. The authors should report results with a stricter protocol that retrieves only the single most similar cluster, and should ablate the 0.9 threshold to show that the improvement is not a threshold artifact.
minor comments (5)
  1. [Fig. 2] The caption contains the typo "Stable hierachy" and would benefit from copy-editing.
  2. [Sec. 3.1] The term "co-visible 3D regions" is not precisely defined; please clarify how N(mi) is constructed and what overlap threshold is used.
  3. [Table 5] The new CoR benchmark is described with only a one-sentence summary in the main text; the annotation procedure, query construction, and evaluation protocol should be described in the main text or the supplementary should be referenced more explicitly.
  4. [Eq. (6)] The notation m* is used before its definition in the following sentence; reorder the derivation for clarity.
  5. [Table 7] N2F2 [2] appears in the supplementary comparison table but is not discussed in the related work; a brief description would help readers interpret the comparison.

Circularity Check

2 steps flagged · score 6.0 of 10

Eq. (1) defines structural levels as quantized physical scales, so the claimed scale-invariant semantic hierarchy reduces to the scale input it claims to transcend.

  1. self definitional [Sec. 3.1, 'Local Peak-based Mask Level Assignment', Eq. (1); Sec. 1, 'Our Insight']
    "we perform peak detection on the histogram of N(mi) ∪ {mi} scales to self-adaptively identify L prominent peaks ... These peaks represent the underlying semantic hierarchy ... Finally, the target mask mi is assigned a discrete level li based on which peak its own scale si aligns with: li = arg min_{l∈{1,...,L}} |si − pl|."

    The 'structural level' is defined as the nearest histogram peak of physical 3D scales, so the level label is a deterministic quantization of the physical-scale input. The paper's headline claim is that these levels are 'strictly invariant to both viewing distances and absolute physical sizes' (Sec. 1). That invariance cannot follow from Eq. (1): two same-rank entities whose 3D diameters fall into different histogram modes receive different level indices, which is exactly the bouquet-flower failure mode the paper uses to motivate the method. The downstream dense supervision, decoupled features, and scene-graph levels all inherit these scale-quantized labels, so the core hierarchy discovery reduces by construction to the size proxy the paper says it abandons.

  2. renaming known result [Sec. 1, 'Our Insight' and Fig. 2(b)-(c); Sec. 3.1, 'Mask Lifting and 3D Scale Estimation']
    "we utilize these physical scales as an initial proxy to establish the semantic hierarchy. By clustering masks based on spatial co-visibility and 3D scale, we effectively discretize the continuous scales into distinct structural levels, thereby manifesting the implicit 3D hierarchy of the scene."

    This is the same scale-based grouping the paper criticizes in Fig. 2(b), where '3D Scale-based methods rely on absolute physical sizes,' with the manual global scale parameter replaced by automatically detected histogram peaks. The '3D Level-based paradigm' is therefore not a distinct mechanism; it is scale-based histogram clustering relabeled as structural levels. The claimed advantage—grouping entities 'regardless of viewing distance or absolute physical size'—is a renaming, not a derived property. The external CLIP grounding keeps the representation useful for open-vocabulary tasks, but the hierarchy-discovery claim itself is equivalent to its scale input.

full rationale

The central circularity is in the definition of the hierarchy: Eq. (1) assigns structural levels by nearest-peak quantization of physical 3D scales, so the paper's 'strictly invariant to absolute physical sizes' claim is not established and is in fact contradicted whenever within-rank scale variance creates separate histogram modes. This is a genuine self-definitional reduction of the core novelty. However, the paper is not wholly circular: the multi-view mask lifting, contrastive distillation with decoupled level features, optimal-view CLIP grounding, and scene-graph construction are independent contributions, and the quantitative results on external benchmarks (NVOS, SPIn-NeRF, LERF-OVS, and 3D-OVS) provide outside evidence that the learned features are useful. The GAGS self-citation (ref. 37) supplies annotations and protocol for Mip-NeRF 360, but that is an evaluation setup rather than a load-bearing derivation step, so it does not by itself raise the score. The score of 6 reflects that one of the paper's principal claims—scale-invariant intrinsic hierarchy discovery—reduces by construction to the physical-scale input, while the remaining pipeline retains substantial independent content.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on the scale-to-level proxy, SAM mask quality, MASt3R-SfM geometry, and CLIP/LLM semantics. Key numbers (L=8, histogram peak parameters, thresholds) are chosen empirically and are not derived.

free parameters (4)
  • Maximum hierarchy depth L = 8
    Set to 8 based on empirical observation that real-world entities rarely decompose beyond 8 levels (Sec. B); the method then trains L=8 feature subspaces regardless of scene.
  • Histogram peak detection parameters = unspecified
    The local peak detection in Sec. 3.1 requires a histogram bandwidth and peak prominence/height thresholds; these are not specified, so the level assignment is underdetermined.
  • Retrieval threshold = 0.9*S_max
    Evaluation protocol in Sec. B uses 0.9*S_max to select matching clusters; this threshold affects reported mIoU/mAcc.
  • Contrastive weight clipping tau_max = unspecified
    Eq. 11 clips inverse-area weights with tau_max; value not reported.
assumptions (5)
  • domain assumption SAM ViT-H produces reliable multi-granular 2D masks at whole/part/subpart levels.
    Entire mask pool M comes from SAM (Sec. 3.1); if SAM fails to segment a part, that part cannot enter the hierarchy.
  • ad hoc to paper Physical 3D scale of a mask is a valid proxy for its semantic structural level.
    Sec. 3.1 uses the effective spatial diameter s_i as the basis for level assignment; this is the central assumption that the paper's 'intrinsic level' claim rests on, yet it is only described as an 'initial proxy'.
  • domain assumption MASt3R-SfM geometry and camera poses are accurate enough for mask lifting and scale estimation.
    Sec. 3 relies on pixel-to-point mapping F from MASt3R-SfM; noise in reconstruction propagates to scale estimates and levels.
  • domain assumption CLIP embeddings align text queries with object appearances.
    Open-vocabulary grounding in Sec. 3.3 uses CLIP features; if CLIP cannot distinguish fine-grained parts, grounding fails.
  • domain assumption Hierarchical axioms (monotonicity, recursive inclusion, structural inheritance) describe real scene semantics.
    These axioms are stated in Sec. 3.1 as requirements for supervision, but for arbitrary scenes a part can appear without a parent mask at a coarser level, or masks may not be nested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LEGO: Leveled Language Gaussian Splatting." pith.science (2026). https://pith.science/paper/4FEBJ6WS

@misc{pith2026260810057,
  author       = {Pith},
  title        = {Pith review of: LEGO: Leveled Language Gaussian Splatting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4FEBJ6WS}},
  note         = {Machine review of arXiv:2608.10057}
}
read the original abstract

We introduce LEGO for advanced open-vocabulary scene understanding. Beyond basic concept recognition, its core innovation lies in capturing the intrinsic semantic hierarchies within the scene, such as the "flowerpot -> bouquet -> bud -> petal" lineage. While foundation models like SAM can identify multi-granular structures in 2D, their partitions are strictly perspective-bound and lack cross-view consensus. LEGO self-adaptively re-grades volatile multi-view SAM granularities into a unified, 3D-consistent hierarchy. This provides precise supervision for the structurally coherent, multi-level segmentation of 3D scenes. By grounding these segments with CLIP embeddings, LEGO recovers open-vocabulary semantic logic across hierarchical levels. Furthermore, by incorporating spatial relationships, we elevate these segments into level-wise language scene graphs, effectively empowering Large Language Models to perform complex, context-aware spatial reasoning and precise visual grounding. Experimental results demonstrate that LEGO establishes new state-of-the-art performance across both promptable and open-vocabulary 3D segmentation benchmarks, exhibiting advanced hierarchical scene decomposition and context-aware spatial reasoning.

Figures

Figures reproduced from arXiv: 2608.10057 by the authors.

Figure 1
Figure 1. LEveled Gaussian splatting for Open-vocabulary scene understand [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of 3D hierarchical segmentation paradigms. (a) 2D [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. LEGO pipeline. (a) LEGO first lifts multiview SAM masks into a 3D coherent scale space to assign them view-consistent structural levels (Sec. 3.1). (b) Guided by these level-masks, a hierarchical contrastive distillation process optimizes the decoupled identity features of the Gaussian field (Sec. 3.2). 3.1 3D Semantic Hierarchy from SAM Masks In this section, we re-grade multi-view, multi-granularity SAM masks into… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Illustration of mask level assignment. Our mechanism discovers the com￾plete 3D hierarchical structure implicit in multi-view 2D masks, assigning each mask to its corresponding view-consistent level. Through this per-mask assignment, as illustrated in [PITH_FULL_IMAGE…
Figure 5
Figure 5. Figure 5: Visualization of automatic hierarchical scene decomposition. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Open-vocabulary segmentation results. While other methods struggle with severe feature blurring, LEGO cleanly isolates extremely fine-grained entities like “corn” and “onion segments”, and preserves exquisite boundaries for “LEGO bonsai”. LLM-Guided Complex Scene Reaso…
Figure 7
Figure 7. Figure 7: Visualization of graph-based Chain-of-Retrieval. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Additional promptable segmentation results in different scenes. [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 9
Figure 9. Figure 9: Additional open-vocabulary segmentation results in [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]
Figure 10
Figure 10. Figure 10: Additional open-vocabulary segmentation results in [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 11
Figure 11. Figure 11: Effectiveness of Graph-Based Chain-of-Retrieval (CoR). [PITH_FULL_IMAGE:figures/full_fig_p028_11.png]
Figure 12
Figure 12. Figure 12: Additional LLM-based chain of retrieval (CoR) results. [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]
Figure 13
Figure 13. Figure 13: Additional LLM-based chain of retrieval (CoR) results. [PITH_FULL_IMAGE:figures/full_fig_p029_13.png]
Figure 14
Figure 14. Figure 14: Additional open-vocabulary segmentation results in [PITH_FULL_IMAGE:figures/full_fig_p030_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 29 canonical work pages

  1. [1]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., Hedman, P.: Mip- nerf 360: Unbounded anti-aliased neural radiance fields. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5470–5479 (2022)

    2022

  2. [2]

    In: European Conference on Computer Vision

    Bhalgat, Y., Laina, I., Henriques, J.F., Zisserman, A., Vedaldi, A.: N2f2: Hierarchi- cal scene understanding with nested neural feature fields. In: European Conference on Computer Vision. pp. 197–214. Springer (2024)

    work page 2024

  3. [3]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9650–9660 (2021)

    2021

  4. [4]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Cen, J., Fang, J., Yang, C., Xie, L., Zhang, X., Shen, W., Tian, Q.: Segment any 3d gaussians. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 1971–1979 (2025)

    work page 2025

  5. [5]

    International Journal of Computer Vision pp

    Cen, J., Fang, J., Zhou, Z., Yang, C., Xie, L., Zhang, X., Shen, W., Tian, Q.: Segment anything in 3d with radiance fields. International Journal of Computer Vision pp. 1–23 (2025)

    work page 2025

  6. [6]

    arXiv preprint arXiv:2505.24746 (2025)

    Cen, J., Zhou, X., Fang, J., Wen, C., Xie, L., Zhang, X., Shen, W., Tian, Q.: Tack- ling view-dependent semantics in 3d language gaussian splatting. arXiv preprint arXiv:2505.24746 (2025)

    arXiv 2025

  7. [7]

    Advances in Neural Information Processing Systems36, 25971–25990 (2023)

    Cen, J., Zhou, Z., Fang, J., Shen, W., Xie, L., Jiang, D., Zhang, X., Tian, Q., et al.: Segment anything in 3d with nerfs. Advances in Neural Information Processing Systems36, 25971–25990 (2023)

    2023

  8. [8]

    arXiv preprint arXiv:2412.01807 (2024)

    Cheng, J., Zaech, J.N., Van Gool, L., Paudel, D.P.: Occam’s lgs: An efficient ap- proach for language gaussian splatting. arXiv preprint arXiv:2412.01807 (2024)

    arXiv 2024

Show all 62 references
  1. [9]

    In: Proceedings of the 33rd ACM International Conference on Multimedia

    Dai, S., Qu, Y., Li, Z., Li, X., Zhang, S., Cao, L.: Training-free hierarchical scene understanding for gaussian splatting with superpoint graphs. In: Proceedings of the 33rd ACM International Conference on Multimedia. pp. 3673–3682 (2025)

    2025

  2. [10]

    In: 2025 International Conference on 3D Vision (3DV)

    Duisterhof, B.P., Zust, L., Weinzaepfel, P., Leroy, V., Cabon, Y., Revaud, J.: Mast3r-sfm: a fully-integrated solution for unconstrained structure-from-motion. In: 2025 International Conference on 3D Vision (3DV). pp. 1–10. IEEE (2025)

    2025

  3. [11]

    In: European conference on computer vision

    Ghiasi,G.,Gu,X.,Cui,Y.,Lin,T.Y.:Scalingopen-vocabularyimagesegmentation with image-level labels. In: European conference on computer vision. pp. 540–557. Springer (2022)

    2022

  4. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Goel, R., Sirikonda, D., Saini, S., Narayanan, P.: Interactive segmentation of ra- diance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4201–4211 (2023)

    2023

  5. [13]

    In: 2024 IEEE Inter- national Conference on Robotics and Automation (ICRA)

    Gu, Q., Kuwajerwala, A., Morin, S., Jatavallabhula, K.M., Sen, B., Agarwal, A., Rivera, C., Paul, W., Ellis, K., Chellappa, R., et al.: Conceptgraphs: Open- vocabulary 3d scene graphs for perception and planning. In: 2024 IEEE Inter- national Conference on Robotics and Automat...

    2024

  6. [14]

    arXiv preprint arXiv:2203.08414 (2022)

    Hamilton, M., Zhang, Z., Hariharan, B., Snavely, N., Freeman, W.T.: Unsuper- vised semantic segmentation by distilling feature correspondences. arXiv preprint arXiv:2203.08414 (2022)

    2022 arXiv

  7. [15]

    arXiv preprint arXiv:2401.17857 (2024) LEGO: Leveled Language Gaussian Splatting 17

    Hu,X.,Wang,Y.,Fan,L.,Luo,C.,Fan,J.,Lei,Z.,Li,Q.,Peng,J.,Zhang,Z.:Sagd: Boundary-enhanced segment anything in 3d gaussian via gaussian decomposition. arXiv preprint arXiv:2401.17857 (2024) LEGO: Leveled Language Gaussian Splatting 17

    2024 arXiv

  8. [16]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops

    Huang, T., Chen, R., Hu, D., Huang, F., Gong, M., Liu, T.: Openinsgaussian: Open-vocabulary instance gaussian segmentation with context-aware cross-view fusion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops. pp. 6400–6409 (October 2025)

    2025

  9. [17]

    Advances in Neural Information Processing Systems37, 89184–89212 (2024)

    Jain, U., Mirzaei, A., Gilitschenski, I.: Gaussiancut: Interactive segmentation via graph cut for 3d gaussian splatting. Advances in Neural Information Processing Systems37, 89184–89212 (2024)

    2024

  10. [18]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Ji, Y., Zhu, H., Tang, J., Liu, W., Zhang, Z., Tan, X., Xie, Y.: Fastlgs: Speeding up language embedded gaussians with feature grid mapping. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 3922–3930 (2025)

    2025

  11. [19]

    splat: Directly referring 3d gaussian splatting via direct language embedding registration

    Jun-Seong, K., Kim, G., Yu-Ji, K., Wang, Y.C.F., Choe, J., Oh, T.H.: Dr. splat: Directly referring 3d gaussian splatting via direct language embedding registration. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 14137–14146 (2025)

    2025

  12. [20]

    ACM Trans

    Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph.42(4), 139–1 (2023)

    2023

  13. [21]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kerr, J., Kim, C.M., Goldberg, K., Kanazawa, A., Tancik, M.: Lerf: Language em- bedded radiance fields. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 19729–19739 (2023)

    2023

  14. [22]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Kim, C.M., Wu, M., Kerr, J., Goldberg, K., Tancik, M., Kanazawa, A.: Garfield: Group anything with radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21530–21539 (2024)

    2024

  15. [23]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4015–4026 (2023)

    2023

  16. [24]

    arXiv preprint arXiv:2201.03546 (2022)

    Li, B., Weinberger, K.Q., Belongie, S., Koltun, V., Ranftl, R.: Language-driven semantic segmentation. arXiv preprint arXiv:2201.03546 (2022)

    2022 arXiv

  17. [25]

    In: Proceedings of the Computer Vision and Pattern Recognition Con- ference

    Li, H., Wu, Y., Meng, J., Gao, Q., Zhang, Z., Wang, R., Zhang, J.: Instance- gaussian: Appearance-semantic joint gaussian representation for 3d instance-level perception. In: Proceedings of the Computer Vision and Pattern Recognition Con- ference. pp. 14078–14088 (2025)

    2025

  18. [26]

    arXiv preprint arXiv:2507.07136 (2025)

    Li, W., Zhao, Y., Qin, M., Liu, Y., Cai, Y., Gan, C., Pfister, H.: Langsplatv2: High-dimensional 3d language gaussian splatting with 450+ fps. arXiv preprint arXiv:2507.07136 (2025)

    2025

  19. [27]

    In: 2025 IEEE International Conference on Robotics and Automation (ICRA)

    Linok, S., Zemskova, T., Ladanova, S., Titkov, R., Yudin, D., Monastyrny, M., Valenkov, A.: Beyond bare queries: Open-vocabulary object grounding with 3d scene graph. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). pp. 13582–13589. IEEE (2025)

    2025

  20. [28]

    Advances in Neural Information Processing Systems36, 53433–53456 (2023)

    Liu, K., Zhan, F., Zhang, J., Xu, M., Yu, Y., El Saddik, A., Theobalt, C., Xing, E., Lu, S.: Weakly supervised 3d open-vocabulary segmentation. Advances in Neural Information Processing Systems36, 53433–53456 (2023)

    2023

  21. [29]

    arXiv preprint arXiv:2404.07977 (2024)

    Lyu, W., Li, X., Kundu, A., Tsai, Y.H., Yang, M.H.: Gaga: Group any gaussians via 3d-aware memory bank. arXiv preprint arXiv:2404.07977 (2024)

    2024 arXiv

  22. [30]

    McInnes, L., Healy, J., Astels, S., et al.: hdbscan: Hierarchical density based clus- tering. J. Open Source Softw.2(11), 205 (2017)

    2017

  23. [31]

    ACM Transactions on Graphics (TOG) (2019)

    Mildenhall, B., Srinivasan, P.P., Ortiz-Cayon, R., Kalantari, N.K., Ramamoorthi, R., Ng, R., Kar, A.: Local light field fusion: Practical view synthesis with prescrip- tive sampling guidelines. ACM Transactions on Graphics (TOG) (2019)

    2019

  24. [32]

    Commu- nications of the ACM65(1), 99–106 (2021) 18 Y

    Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. Commu- nications of the ACM65(1), 99–106 (2021) 18 Y. Peng et al

    2021

  25. [33]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition

    Mirzaei, A., Aumentado-Armstrong, T., Derpanis, K.G., Kelly, J., Brubaker, M.A., Gilitschenski, I., Levinshtein, A.: Spin-nerf: Multiview segmentation and percep- tual inpainting with neural radiance fields. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and P...

    2023

  26. [34]

    arXiv preprint arXiv:2304.07193 (2023)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

    2023 arXiv

  27. [35]

    arXiv preprint arXiv:2410.07577 (2024)

    Peng, Q., Planche, B., Gao, Z., Zheng, M., Choudhuri, A., Chen, T., Chen, C., Wu, Z.: 3d vision-language gaussian splatting. arXiv preprint arXiv:2410.07577 (2024)

    2024 arXiv

  28. [36]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Peng, S., Genova, K., Jiang, C., Tagliasacchi, A., Pollefeys, M., Funkhouser, T., et al.: Openscene: 3d scene understanding with open vocabularies. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 815– 824 (2023)

    2023

  29. [37]

    arXiv preprint arXiv:2412.13654 (2024)

    Peng, Y., Wang, H., Liu, Y., Wen, C., Dong, Z., Yang, B.: Gags: Granularity- aware feature distillation for language gaussian splatting. arXiv preprint arXiv:2412.13654 (2024)

    2024 arXiv

  30. [38]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Qin, M., Li, W., Zhou, J., Wang, H., Pfister, H.: Langsplat: 3d language gaussian splatting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20051–20060 (2024)

    2024

  31. [39]

    In: Proceedings of the 32nd ACM international conference on multimedia

    Qu, Y., Dai, S., Li, X., Lin, J., Cao, L., Zhang, S., Ji, R.: Goi: Find 3d gaussians of interest with an optimizable open-vocabulary semantic-space hyperplane. In: Proceedings of the 32nd ACM international conference on multimedia. pp. 5328– 5337 (2024)

    2024

  32. [40]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Ren, Z., Agarwala, A., Russell, B., Schwing, A.G., Wang, O.: Neural volumetric object selection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 6133–6142 (2022)

    2022

  33. [41]

    In: European Conference on Computer Vision

    Shen, Q., Yang, X., Wang, X.: Flashsplat: 2d to 3d gaussian splatting segmenta- tion solved optimally. In: European Conference on Computer Vision. pp. 456–472. Springer (2024)

    2024

  34. [42]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Shen, Y., Fu, C., Chen, P., Zhang, M., Li, K., Sun, X., Wu, Y., Lin, S., Ji, R.: Aligning and prompting everything all at once for universal visual perception. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13193–13203 (2024)

    2024

  35. [43]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition

    Shi, J.C., Wang, M., Duan, H.B., Guan, S.H.: Language embedded 3d gaussians for open-vocabulary scene understanding. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition. pp. 5333–5343 (2024)

    2024

  36. [44]

    arXiv preprint arXiv:2504.11893 (2025)

    Sun, W., Zhou, Y., Jiao, J., Li, Y.: Cags: Open-vocabulary 3d scene understanding with context-aware gaussian splatting. arXiv preprint arXiv:2504.11893 (2025)

    2025 arXiv

  37. [45]

    arXiv preprint arXiv:2504.08581 (2025)

    Tan, X., Ji, Y., Zhu, H., Xie, Y.: Fmlgs: Fast multilevel language embedded gaus- sians for part-level interactive agents. arXiv preprint arXiv:2504.08581 (2025)

    2025 arXiv

  38. [46]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Wang, H., Liu, Y., Liu, Z., Wang, W., Dong, Z., Yang, B.: Vistadream: Sampling multiview consistent images for single-view scene reconstruction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 26772–26782 (2025)

    2025

  39. [47]

    arXiv preprint arXiv:2503.04034 (2025)

    Wang, X., Yang, D., Gao, Y., Yue, Y., Yang, Y., Fu, M.: Gaussiangraph: 3d gaussian-based scene graph generation for open-world scene understanding. arXiv preprint arXiv:2503.04034 (2025)

    2025 arXiv

  40. [48]

    In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision

    Wang, Z., Su, Y., Li, C., Wang, D., Huang, Y., Li, X., Zhao, B.: Open-vocabulary octree-graph for 3d scene understanding. In: Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision. pp. 7037–7047 (2025) LEGO: Leveled Language Gaussian Splatting 19

    2025

  41. [49]

    In: First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024 (2024)

    Werby, A., Huang, C., Büchner, M., Valada, A., Burgard, W.: Hierarchical open- vocabulary 3d scene graphs for language-grounded robot navigation. In: First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024 (2024)

    2024

  42. [50]

    Advances in Neural Information Processing Systems 37, 19114–19138 (2024)

    Wu, Y., Meng, J., Li, H., Wu, C., Shi, Y., Cheng, X., Zhao, C., Feng, H., Ding, E., Wang, J., et al.: Opengaussian: Towards point-level 3d gaussian-based open vocabulary understanding. Advances in Neural Information Processing Systems 37, 19114–19138 (2024)

    2024

  43. [51]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Yang,Z.,Yang,B.,Dong,W.,Cao,C.,Cui,L.,Ma,Y.,Cui,Z.,Bao,H.:Instascene: Towards complete 3d instance decomposition and reconstruction from cluttered scenes. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 7771–7781 (2025)

    2025

  44. [52]

    In: European conference on computer vision

    Ye, M., Danelljan, M., Yu, F., Ke, L.: Gaussian grouping: Segment and edit any- thing in 3d scenes. In: European conference on computer vision. pp. 162–179. Springer (2024)

    2024

  45. [53]

    Journal of Machine Learning Research26(34), 1–17 (2025)

    Ye, V., Li, R., Kerr, J., Turkulainen, M., Yi, B., Pan, Z., Seiskari, O., Ye, J., Hu, J., Tancik, M., et al.: gsplat: An open-source library for gaussian splatting. Journal of Machine Learning Research26(34), 1–17 (2025)

    2025

  46. [54]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Ying, H., Yin, Y., Zhang, J., Wang, F., Yu, T., Huang, R., Fang, L.: Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20612–20622 (2024)

    2024

  47. [55]

    arXiv preprint arXiv:2506.06822 (2025)

    Zhan, C., Zhang, Y., Wang, G., Wang, H.: Hi-lsplat: Hierarchical 3d language gaussian splatting. arXiv preprint arXiv:2506.06822 (2025)

    2025 arXiv

  48. [56]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Zhang,J.,Jiang,J.,Chen,Y.,Jiang,K.,Liu,X.:Cob-gs:Clearobjectboundariesin 3dgs segmentation based on boundary-adaptive gaussian splitting. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 19335–19344 (2025)

    2025

  49. [57]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Zhao, Y., Xu, W., Zheng, R., Qiao, P., Liu, C., Chen, J.: isegman: Interactive segment-and-manipulate 3d gaussians. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 661–670 (2025)

    2025

  50. [58]

    arXiv preprint arXiv:2606.08980 (2026)

    Zhu, R., Guo, J., Guo, X., Liu, Z., Hui, K.H., Yin, W., Chen, K., Chen, W., Ren, W., Liu, Y., et al.: Eps3d: End-to-end feed-forward 3d panoptic segmentation. arXiv preprint arXiv:2606.08980 (2026)

    2026 arXiv

  51. [59]

    arXiv preprint arXiv:2510.20238 (2025)

    Zhu, R., Hui, K.H., Liu, Z., Wu, Q., Tang, W., Qiu, S., Heng, P.A., Fu, C.W.: Cos3d: Collaborative open-vocabulary 3d segmentation. arXiv preprint arXiv:2510.20238 (2025)

    2025

  52. [60]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhu, R., Qiu, S., Liu, Z., Hui, K.H., Wu, Q., Heng, P.A., Fu, C.W.: Rethinking end-to-end 2d to 3d scene segmentation in gaussian splatting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3656–3665 (2025)

    2025

  53. [61]

    In: European Conference on Computer Vision

    Zhu, R., Qiu, S., Wu, Q., Hui, K.H., Heng, P.A., Fu, C.W.: Pcf-lift: Panoptic lifting by probabilistic contrastive fusion. In: European Conference on Computer Vision. pp. 92–108. Springer (2024)

    2024

  54. [62]

    Find the han- dle of the pitcher beside the rolling pin

    Zuo, X., Samangouei, P., Zhou, Y., Di, Y., Li, M.: Fmgs: Foundation model em- bedded 3d gaussian splatting for holistic 3d scene understanding. International Journal of Computer Vision133(2), 611–627 (2025) 20 Y. Peng et al. Supplementary Material The supplementary material is...

    2025

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.