Pith. sign in

REVIEW 4 major objections 4 minor 43 references

DiSCO-3D : Discovering and segmenting Sub-Concepts from Open-vocabulary queries in NeRF

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that 3D open-vocabulary sub-concept discovery can be solved by running unsupervised prototype clustering and weak CLIP guidance jointly inside a NeRF feature field, producing segmentations that adapt to both the scene and…

desk verdict New task and a sensible joint segmentation method, but the 'discovery' claim hinges on an Nq that the paper never specifies. read the letter →

arxiv 2507.14596 v1 pith:LRJG3MND submitted 2025-07-19 cs.CV

Doriand Petit , Steve Bourgeois , Vincent Gay-Bellile , Florian Chabot , Loïc Barthe This is my paper · ORCID
classification cs.CV
keywords 3Dsemanticsegmentationopen-vocabularyunsupervisedneuralradiancefieldsfeatureprototypeclusteringsub-conceptdiscoveryCLIP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a new 3D segmentation task, Open-Vocabulary Sub-concepts Discovery (OV-SD), in which a user gives a query such as "furniture" but does not enumerate the classes to return; the model must decompose the scene into the query's sub-concepts that are actually present. DiSCO-3D is presented as the first method for this task, built on a pretrained NeRF feature field that provides both a spatially precise semantic feature (DINO) and a queryable open-vocabulary feature (CLIP). The core proposal is to run unsupervised prototype-based clustering and weak query guidance jointly rather than successively, so irrelevant regions are discarded while the remaining scene is split into discoverable sub-concepts. If correct, a single 3D representation serves both open-vocabulary segmentation and unsupervised segmentation, with per-query optimization of about two seconds.

What carries the argument

The central object is a set of learnable prototypes $\{P_i\}_{i=1}^N$ in the projected feature space, updated by an exponential moving average that weights samples by both NeRF density weights $w_k$ and prediction confidence $D_{k,i}$. Projected DINO features are assigned to prototypes through a softmax over cosine similarity with temperature $\beta$, which decays linearly to sharpen clusters. Three losses carry the argument: $\mathcal{L}_{\mathrm{proj}}$ preserves DINO-space correspondence relations, $\mathcal{L}_{\mathrm{irr}}^q$ uses the CLIP field's query relevance mask to push query-irrelevant samples toward irrelevant prototypes while keeping relevant samples on relevant prototypes, and $\mathcal{L}_{\mathrm{proto}}$ pulls each projected DINO feature toward the prototype whose CLIP embedding is most similar to the sample's CLIP embedding. A parallel set of CLIP prototypes, updated with the same EMA, gives every discovered sub-concept a nameable embedding, which is what makes the unsupervised clustering and open-vocabulary guidance happen inside one optimization.

What would settle it

Take a scene/query pair whose ground-truth sub-concept count is withheld, run DiSCO-3D with a deliberately wrong prototype budget (say three when the scene contains seven ground-truth sub-concepts), and measure PQ and mIoU against the benchmark; if performance collapses and the model cannot signal that the budget is wrong, then the method's discovery behavior depends on an oracle count rather than on the scene itself.

Watch

Extended reading notes

Core claim

The central claim is that 3D Open-Vocabulary Sub-concepts Discovery is a distinct, solvable problem and that DiSCO-3D solves it: given a pretrained feature field and one or more CLIP-encoded queries, the method jointly learns a projector over DINO features and a set of prototypes, supervised so that prototypes split into query-relevant and query-irrelevant groups. The relevant prototypes define the discovered sub-concepts, and each carries a CLIP prototype embedding that supports a posteriori naming and matching. The authors assert that this joint formulation beats the natural sequential baselines (open-vocabulary segmentation followed by unsupervised clustering, or the reverse) on a new Replica-based benchmark, and that in the edge cases of single-concept queries and no-query unsupervised segmentation, DiSCO-3D improves on or matches the best available results.

Load-bearing premise

The method assumes the user or benchmark says in advance how many relevant sub-concepts to look for; if that count is unknown or wrong, the discovery formulation has no built-in way to correct itself.

Editorial extensions

If this is right

  • Plugging DiSCO-3D into LeRF or OpenNeRF improves open-vocabulary segmentation on Replica, raising mIoU and mAcc over the raw feature fields by filling relevancy holes and reducing relevancy spilling.
  • In the no-query edge case, DiSCO-3D outperforms K-Means on the same feature field as well as the 2D and point-cloud unsupervised segmentation baselines SmooSeg and GrowSP.
  • The same one-hot formulation handles multiple simultaneous queries, including disjoint, overlapping, and nested queries, without extra constraints.
  • A new query only re-optimizes the projector and prototypes, converging in about two seconds on a single RTX 4090 after the feature field is pretrained.
  • The architecture extends to alternative segmentation fields such as a SAM-based feature field and to alternative query modalities such as visual embeddings or user clicks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step the authors leave open is estimating the number of sub-concepts automatically; until such an estimator exists, the discovery claim depends on the user supplying a prototype budget, which the paper sets from ground-truth counts in its main ablations.
  • Because each sub-concept gets a nameable CLIP prototype, the method could serve as an interactive labeler: a user could query "tools," inspect the discovered names, and re-query at finer granularity without retraining the feature field.
  • The joint-training design transfers in principle to other continuous 3D representations such as Gaussian Splatting or to 2D images, with NeRF density weights replaced by a comparable per-sample confidence measure; the paper notes the extension is theoretical.
  • The benchmark's groupings contain semantically overlapping classes such as armchair, sofa, and chair, so a well-calibrated prototype should reflect graded ambiguity rather than a single hard label; the paper's reported matching errors suggest this is where the method's performance ceiling lies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces 3D Open-Vocabulary Sub-concepts Discovery (OV-SD), a segmentation task in which a NeRF-based 3D scene is decomposed into sub-concepts relevant to a user query without the user naming the classes. The proposed method, DiSCO-3D, combines a prototype-based unsupervised semantic segmentation module trained on DINO features extracted from a pretrained feature field (LeRF or OpenNeRF) with weak supervision from CLIP query relevance, via the losses Lproj, Lirr, and Lproto. The authors also introduce a Replica-based benchmark with 40 LLM-generated concepts and compare their method against sequential OVSeg-to-USS and USS-to-OVSeg baselines. They report that DiSCO-3D outperforms these baselines on PQ, mIoU, and mAcc for both LeRF and OpenNeRF, and that it also improves the edge cases of open-vocabulary segmentation and unsupervised semantic segmentation.

Significance. If the results hold, the paper makes a useful contribution: it defines a new task that sits between open-vocabulary and unsupervised segmentation, it adapts prototype-based clustering to neural fields, and it provides a benchmark and a consistent joint-versus-sequential comparison that controls for input features and supervision. The qualitative demonstrations across multiple feature fields and query types are informative. A key part of the claimed contribution, however, is the word 'discovery,' and that claim currently rests on an unstated choice of the number of relevant prototypes, Nq, which is load-bearing for the method's central promise.

major comments (4)
  1. [Section 3.4, Eq. (4)] The loss Lq_irr is defined through the one-hot vector Hq, which has Nq leading ones, so the training signal requires Nq as an input. The paper does not specify how Nq should be obtained from a user query alone, and Section 3.5 explicitly states that relevant prototypes must be defined a priori. In the main OV-SD benchmark (Section 4.2.1), the total number of prototypes is fixed to N=10, but the actual Nq used to construct Hq is never reported. If Nq comes from ground-truth sub-concept counts, the benchmark evaluates clustering with a known k and the discovery claim is weakened; if, as suggested by the supplementary hyperparameter section, Nirr=3 and hence Nq=7, that choice should be stated explicitly and its effect on queries with up to 9 ground-truth sub-concepts should be analyzed. This is a load-bearing point for the central contribution and should be resolved by either proposing an estimator for Nq or evaluating the method in a setting where Nq is not oracle-derived.
  2. [Section 4.2.3, Table 2] The robustness claim in Table 2 is entangled with the same ambiguity. The ablation parameterizes N as NGT+Nadd, so it demonstrates insensitivity to excess prototypes only when NGT is known. The last column, N=10, is meant to avoid ground-truth knowledge, but without reporting how Hq is constructed the claim is not verifiable. Please report the exact values of Nq and Nirr, and describe how the first Nq prototype indices are selected, both in the ablations and in the main benchmark.
  3. [Section 4.2.2, Table 1] The quantitative comparison that supports the central 'joint better than sequential' claim is reported without error bars, variance, or number of runs. The absolute PQ values are small (e.g., 8.13 vs. 5.99 on LeRF), so seed-to-seed variability could affect the conclusions. The authors should report standard deviations across multiple runs, or at least justify why the margins are robust.
  4. [Section 4.2.1, Table 1] The K-Means baselines are missing the PCLIP columns in Table 1, making the comparison incomplete under the matching protocol that uses CLIP prototypes. The authors should either explain why these baselines cannot be evaluated under the same protocol, or provide the missing numbers.
minor comments (4)
  1. [Section 4.2.1] The statement 'no concept query exceeds 9 ground-truth sub-concepts' is indirect; please report the distribution of sub-concept counts in the benchmark so that the reader can judge whether N=10 is a safe margin.
  2. [Equation (4)] The normalization in Eq. (4) appears to use #Mq in both terms, although the second sum is over the complement of Mq. Please check whether the second denominator should be the cardinality of the complement, and clarify the notation for Mq.
  3. [Section 3.3] The role of the 'Relevant / Irrelevant Prototypes' block in Figure 2 is not explained until Section 3.4; adding a forward reference would improve readability.
  4. [Table 2] The row label 'Used Nadd' is defined only in the table caption; please define it explicitly in the main text before the table is referenced.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the OV-SD derivation and benchmark are self-contained; the Nq/Hq uncertainty is a reproducibility gap, not a construction-level reduction.

full rationale

DiSCO-3D's claimed derivation chain is (i) cluster DINO features from a pretrained feature field into prototypes (Eqs. 1-3), (ii) use a CLIP-field query mask Mq as weak supervision to separate query-relevant from irrelevant prototypes (Eq. 4), and (iii) optionally regularize with CLIP prototypes (Eq. 5). None of these losses is equivalent to the reported evaluation metric. The benchmark compares DiSCO-3D against naive sequential baselines that share the same feature fields and hyperparameters, using an externally defined Replica-based concept dataset; the reported PQ/mIoU/mAcc scores are not forced by the loss definitions. The only potentially circular element is Hq in Eq. 4, which requires the number Nq of relevant prototypes to be fixed in advance. However, the paper fixes total prototype count once (N=10) and states in the supplementary that three irrelevant prototypes are used in all experiments, so Hq is a fixed model hyperparameter rather than a per-query ground-truth count; Table 2's NGT-anchored rows are explicitly robustness ablations whose final 'N=10' column is intended to be GT-independent. Whether that choice is adequately documented and whether it limits concepts with up to nine GT sub-concepts is a correctness/reproducibility concern, not a self-referential reduction. The CLIP-prototype matching used for evaluation shares the CLIP embedding space that provides weak supervision, but CLIP is a fixed pretrained external model and the paper also reports Hungarian (GT-mask) matching, so the evaluation is not re-discovering the training target by construction. No load-bearing self-citation chain appears. I therefore find no circular step.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on two feature-field quality assumptions (DINO spatial semantics, CLIP query relevancy) and on the a priori specification of Nq. The free parameters are standard clustering/optimization hyperparameters, but τ and Nq are directly tied to the benchmark and could inflate results if chosen using ground truth.

free parameters (6)
  • CLIP relevancy threshold τ = 0.5 (LeRF), 0.55 (OpenNeRF)
    Defines the binary mask Mq that supervises Lirr in Eq. 4; the supplement calls it the most crucial hyperparameter and states it depends on the feature field.
  • Number of relevant prototypes Nq = Implied to equal GT sub-concept count; total N=10, with at most 9 GT sub-concepts per query
    Sets how many sub-concepts the model can output; the one-hot Hq in Eq. 4 requires a priori definition of relevant prototypes; ablations use N=NGT+Nadd.
  • Sharpness parameter β schedule = 0.5 decreasing linearly to 0.1
    Controls softmax sharpness in Eq. 1; chosen by hand and annealed to enforce cluster separation.
  • Loss weights wproj, wirr, wproto = 20, 1, 0.5
    Balance correlation, irrelevancy, and prototype losses; tuned in experiments (Supp. §7.1).
  • Correlation threshold b for Lproj = 0.5
    Decides which sample pairs are close/distant in DINO space in Eq. 3.
  • EMA decay α and sample filtering thresholds = α=0.998; filter if Dk,i<0.2 or wk<0.2
    Stabilize prototype updates and remove low-confidence/low-density samples; chosen by hand.
assumptions (5)
  • domain assumption DINO feature fields from LeRF encode spatially precise semantic information suitable for prototype clustering
    Invoked in §3.3 as input to the projector; if DINO features do not align with semantics, clustering would not produce meaningful classes.
  • domain assumption CLIP feature field similarities reliably indicate query-relevant regions when thresholded
    The mask Mq in Eq. 4 directly supervises the projector; the supplement's Limitations section shows failures for queries like 'Eggs' and 'Art', confirming the load-bearing nature.
  • ad hoc to paper The number of sub-concepts Nq is known or specifiable a priori
    Supervision Eq. 4 requires choosing which prototypes are relevant; the benchmark appears to set this from GT counts, but the paper does not state how a user would choose Nq in practice.
  • domain assumption NeRF density weights wi indicate the relevance of each 3D sample for feature aggregation
    Used in the two-fold weighted EMA update in Eq. 2; samples in free space or inside objects are downweighted.
  • domain assumption Replica annotations and LLM-generated concepts are valid ground truth for sub-concept segmentation
    The new benchmark labels 40 concepts across 8 scenes by grouping Replica classes; the groups are not human-validated beyond the LLM prompt.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiSCO-3D : Discovering and segmenting Sub-Concepts from Open-vocabulary queries in NeRF." pith.science (2026). https://pith.science/paper/LRJG3MND

@misc{pith2026250714596,
  author       = {Pith},
  title        = {Pith review of: DiSCO-3D : Discovering and segmenting Sub-Concepts from Open-vocabulary queries in NeRF},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LRJG3MND}},
  note         = {Machine review of arXiv:2507.14596}
}
read the original abstract

3D semantic segmentation provides high-level scene understanding for applications in robotics, autonomous systems, \textit{etc}. Traditional methods adapt exclusively to either task-specific goals (open-vocabulary segmentation) or scene content (unsupervised semantic segmentation). We propose DiSCO-3D, the first method addressing the broader problem of 3D Open-Vocabulary Sub-concepts Discovery, which aims to provide a 3D semantic segmentation that adapts to both the scene and user queries. We build DiSCO-3D on Neural Fields representations, combining unsupervised segmentation with weak open-vocabulary guidance. Our evaluations demonstrate that DiSCO-3D achieves effective performance in Open-Vocabulary Sub-concepts Discovery and exhibits state-of-the-art results in the edge cases of both open-vocabulary and unsupervised segmentation.

Figures

Figures reproduced from arXiv: 2507.14596 by the authors.

Figure 1
Figure 1. We introduce the 3D Open-Vocabulary Sub-concepts Discovery (OV-SD) paradigm, which aims to provide a 3D semantic [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of DiSCO-3D for a LeRF Feature Field. DiSCO-3D inputs pairs of features from 3D samples into a projector net￾work learnt to accentuate semantic disparities. Those projected features are then classified by comparing them to class-specific prototypes (subsection 3.3). Thoses prototypes are updated each epoch using an EMA with the projected features. A user query can be used to super￾vise the projector by enco… view at source ↗
Figure 3
Figure 3. DiSCO-3D Qualitative Evaluation for OV-SD. We present results for various queries, scenes (which originate from [12, 21, 33]) and feature fields (LeRF in orange and OpenNeRF blue). (b), (e) and (f) illustrate multiple queries, resp. disjoint, overlapping and nested, (g) a visual query encoded with CLIP and (h) a CLIP feature obtained by a user click as query. Finally, (i) and (j) are OV-SD edge cases, where (i) has … view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Linking Sub-concepts to a posteriori Textual Classes. The queries of the left and right images are respectively ”Sleep” and ”Furniture”. By comparing each CLIP prototype to Replica’s semantic classes encoded with CLIP, DiSCO-3D is able to choose the most relevant class…
Figure 5
Figure 5. Figure 5: Projector Architecture. 6. DiSCO-3D 6.1. Additional Architecture Details Some architecture details and minor contributions have been overlooked in the main paper that we want to cover here. Projector Architecture. Although some USS methods implement a simple linear MLP…
Figure 6
Figure 6. Figure 6: Segmentation Confidence of DiSCO-3D. The query is ”door” [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Limitation #1. By querying ”Eggs”, the LeRF and OpenNeRF baselines makes different prediction, both regarding the responding objects and their precision. While DiSCO-3D can ”repair” segmentation imprecision via the DINO features, it is de￾pendent of the open-vocabulary…
Figure 9
Figure 9. Figure 9: Optimization Timelapse. In average, one epoch takes 22ms, resulting in a training of 200 epochs in ∼ 4s. The query is ”furniture” [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: SAM Feature Field. We replace the DINO feature field in LeRF by a SAM feature field and demonstrate its capacity to perform USS, OV-Seg and OV-SD [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Garfield Feature Field. We use Garfield (SAM Masks outputs) as the segmentation field and perform USS. Note that Garfield being an instance feature field, it cannot be used as a replacement for DINO to perform OVSeg and OV-SD. USS also cannot be entirely considered as…
Figure 12
Figure 12. Figure 12: Effect of the Regularization Loss. Adding the regu￾larization loss reduces over-segmenting (ie. describing single ob￾jects with more than one prototype). The query is ”furniture”. Lproto Nadd 0 2 5 10 20 N = 10 ✓ Used Nadd -0.12 1.08 1.52 1.91 1.96 1.80 PQ ↑ 8.53 9.52…
Figure 13
Figure 13. Figure 13: Top-10 Class Labels Linking for every Sub-Concepts. The query is ”furniture” [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: OV-SD Example Naive Baselines vs DiSCO-3D. The query is ”light” [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: Additional OVSeg Results [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 17
Figure 17. Figure 17: Example of USS on real data [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 34 canonical work pages

  1. [1]

    Deep se- mantic segmentation of natural and medical images: a re- view

    Saeid Asgari Taghanaki, Kumar Abhishek, Joseph Paul Co- hen, Julien Cohen-Adad, and Ghassan Hamarneh. Deep se- mantic segmentation of natural and medical images: a re- view. Artificial Intelligence Review, 54:137–178, 2021. 1

    work page 2021

  2. [2]

    Mip-nerf 360: Unbounded anti-aliased neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5470–5479, 2022. 5

    work page 2022

  3. [3]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021. 2, 3

    work page 2021

  4. [4]

    Interactive segment anything nerf with fea- ture imitation

    Xiaokang Chen, Jiaxiang Tang, Diwen Wan, Jingbo Wang, and Gang Zeng. Interactive segment anything nerf with fea- ture imitation. arXiv preprint arXiv:2305.16233, 2023. 2

    arXiv 2023

  5. [5]

    Selective visual repre- sentations improve convergence and generalization for em- bodied ai

    Ainaz Eftekhar, Kuo-Hao Zeng, Jiafei Duan, Ali Farhadi, Ani Kembhavi, and Ranjay Krishna. Selective visual repre- sentations improve convergence and generalization for em- bodied ai. In ICLR, 2024. 2

    work page 2024

  6. [6]

    Opennerf: Open set 3d neural scene segmentation with pixel- wise features and rendered novel views

    Francis Engelmann, Fabian Manhardt, Michael Niemeyer, Keisuke Tateno, Marc Pollefeys, and Federico Tombari. Opennerf: Open set 3d neural scene segmentation with pixel- wise features and rendered novel views. arXiv preprint arXiv:2404.03650, 2024. 8, 7

    arXiv 2024

  7. [7]

    Deep multi-modal object de- tection and semantic segmentation for autonomous driving: Datasets, methods, and challenges

    Di Feng, Christian Haase-Sch ¨utz, Lars Rosenbaum, Heinz Hertlein, Claudius Glaeser, Fabian Timm, Werner Wies- beck, and Klaus Dietmayer. Deep multi-modal object de- tection and semantic segmentation for autonomous driving: Datasets, methods, and challenges. IEEE Transactions on Intelligent Transportation Systems, 22(3):1341–1360, 2020. 1

    work page 2020

  8. [8]

    Scal- ing open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scal- ing open-vocabulary image segmentation with image-level labels. In European Conference on Computer Vision, pages 540–557. Springer, 2022. 2, 3, 1

    work page 2022

Show all 43 references
  1. [9]

    Unsupervised semantic segmentation by distilling feature correspondences

    Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, and William T Freeman. Unsupervised semantic segmentation by distilling feature correspondences. arXiv preprint arXiv:2203.08414, 2022. 2, 4, 1

    2022 arXiv

  2. [10]

    Semantic scene segmentation for robotics

    Juana Valeria Hurtado and Abhinav Valada. Semantic scene segmentation for robotics. In Deep learning for robot perception and cognition, pages 279–311. Elsevier, 2022. 1

    2022

  3. [11]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42 (4), 2023. 2

    2023

  4. [12]

    Lerf: Language embedded radiance fields

    Justin* Kerr, Chung Min* Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In International Conference on Computer Vision (ICCV), 2023. 2, 3, 6, 7, 8

    2023

  5. [13]

    Eagle: Eigen aggregation learning for object-centric unsupervised semantic segmentation

    Chanyoung Kim, Woojung Han, Dayun Ju, and Seong Jae Hwang. Eagle: Eigen aggregation learning for object-centric unsupervised semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3523–3533, 2024. 2, 4

    2024

  6. [14]

    Garfield: Group anything with radiance fields

    Chung Min Kim, Mingxuan Wu, Justin Kerr, Ken Gold- berg, Matthew Tancik, and Angjoo Kanazawa. Garfield: Group anything with radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21530–21539, 2024. 2

    2024

  7. [15]

    Panoptic segmentation

    Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Doll ´ar. Panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9404–9413, 2019. 6

    2019

  8. [16]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015–4026, 2023. 2

    2023

  9. [17]

    Decomposing nerf for editing via feature field distil- lation

    Sosuke Kobayashi, Eiichi Matsumoto, and Vincent Sitz- mann. Decomposing nerf for editing via feature field distil- lation. Advances in Neural Information Processing Systems, 35:23311–23330, 2022. 2

    2022

  10. [18]

    Smooseg: smoothness prior for unsupervised semantic segmentation

    Mengcheng Lan, Xinjiang Wang, Yiping Ke, Jiaxing Xu, Litong Feng, and Wayne Zhang. Smooseg: smoothness prior for unsupervised semantic segmentation. Advances in Neural Information Processing Systems, 36, 2024. 2, 4, 8, 1

    2024

  11. [19]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation. ICLR, 2022. 2, 1

    2022

  12. [20]

    Acseg: Adaptive conceptualization for unsupervised semantic seg- mentation

    Kehan Li, Zhennan Wang, Zesen Cheng, Runyi Yu, Yian Zhao, Guoli Song, Chang Liu, Li Yuan, and Jie Chen. Acseg: Adaptive conceptualization for unsupervised semantic seg- mentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7162–7172,

  13. [21]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pa...

    2024

  14. [22]

    U3ds3: Unsupervised 3d semantic scene segmenta- tion

    Jiaxu Liu, Zhengdi Yu, Toby P Breckon, and Hubert PH Shum. U3ds3: Unsupervised 3d semantic scene segmenta- tion. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3759–3768, 2024. 2

    2024

  15. [23]

    Weakly supervised 3d open- vocabulary segmentation

    Kunhao Liu, Fangneng Zhan, Jiahui Zhang, Muyu Xu, Yingchen Yu, Abdulmotaleb El Saddik, Christian Theobalt, Eric Xing, and Shijian Lu. Weakly supervised 3d open- vocabulary segmentation. Advances in Neural Information Processing Systems, 36:53433–53456, 2023. 2

    2023

  16. [24]

    Sanerf-hq: Segment anything for nerf in high quality

    Yichen Liu, Benran Hu, Chi-Keung Tang, and Yu-Wing Tai. Sanerf-hq: Segment anything for nerf in high quality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3216–3226, 2024. 2 9

    2024

  17. [25]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 2, 3

    2021

  18. [26]

    Review the state-of-the-art technologies of semantic segmentation based on deep learning.Neurocomputing, 493: 626–646, 2022

    Yujian Mo, Yan Wu, Xinneng Yang, Feilin Liu, and Yujun Liao. Review the state-of-the-art technologies of semantic segmentation based on deep learning.Neurocomputing, 493: 626–646, 2022. 1

    2022

  19. [27]

    Instant neural graphics primitives with a multires- olution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a multires- olution hash encoding. ACM Trans. Graph., 41(4):102:1– 102:15, 2022. 3

    2022

  20. [28]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2, 3

    2023 arXiv

  21. [29]

    Openscene: 3d scene understanding with open vocabularies

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 815–824, 2023. 2

    2023

  22. [30]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  23. [31]

    Distilled feature fields en- able few-shot language-guided manipulation

    William Shen, Ge Yang, Alan Yu, Jansen Wong, Leslie Pack Kaelbling, and Phillip Isola. Distilled feature fields en- able few-shot language-guided manipulation. arXiv preprint arXiv:2308.07931, 2023. 2

    2023 arXiv

  24. [32]

    Language embedded 3d gaussians for open- vocabulary scene understanding

    Jin-Chuan Shi, Miao Wang, Hao-Bin Duan, and Shao- Hua Guan. Language embedded 3d gaussians for open- vocabulary scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5333–5343, 2024. 2, 8

    2024

  25. [33]

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler Gi...

    1906 arXiv

  26. [34]

    Nerfstudio: A modular framework for neural radiance field development

    Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, et al. Nerfstudio: A modular framework for neural radiance field development. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1–12, 2023. 5

    2023

  27. [35]

    Neural feature fusion fields: 3d distillation of self- supervised 2d image representations

    Vadim Tschernezki, Iro Laina, Diane Larlus, and Andrea Vedaldi. Neural feature fusion fields: 3d distillation of self- supervised 2d image representations. In 2022 International Conference on 3D Vision (3DV), pages 443–453. IEEE,

    2022

  28. [36]

    Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation

    Monika Wysocza ´nska, Oriane Sim´eoni, Micha¨el Ramamon- jisoa, Andrei Bursuc, Tomasz Trzci ´nski, and Patrick P ´erez. Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation. ECCV, 2024. 2, 1

    2024

  29. [37]

    Growsp: Unsupervised semantic segmentation of 3d point clouds

    Zihui Zhang, Bo Yang, Bing Wang, and Bo Li. Growsp: Unsupervised semantic segmentation of 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17619–17629, 2023. 2, 8

    2023

  30. [38]

    Open-vocabulary uni- versal image segmentation with maskclip

    Zhuowen Tu Zheng Ding, Jieke Wang. Open-vocabulary uni- versal image segmentation with maskclip. In International Conference on Machine Learning, 2023. 2, 1

    2023

  31. [39]

    In-place scene labelling and understanding with implicit scene representation

    Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and An- drew Davison. In-place scene labelling and understanding with implicit scene representation. In Proceedings of the International Conference on Computer Vision (ICCV), 2021. 8

    2021

  32. [40]

    Supervised semantic segmentation based on deep learning: a survey

    Yuguo Zhou, Yanbo Ren, Erya Xu, Shiliang Liu, and Li- jian Zhou. Supervised semantic segmentation based on deep learning: a survey. Multimedia Tools and Applications, 81 (20):29283–29304, 2022. 2

    2022

  33. [41]

    Generalized decoding for pixel, image, and language

    Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Lu Yuan, et al. Generalized decoding for pixel, image, and language. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1511...

    2023

  34. [42]

    Additional Architecture Details Some architecture details and minor contributions have been overlooked in the main paper that we want to cover here

    DiSCO-3D 6.1. Additional Architecture Details Some architecture details and minor contributions have been overlooked in the main paper that we want to cover here. Projector Architecture. Although some USS methods implement a simple linear MLP projector [9], we follow SmooSeg [...

  35. [43]

    Hyperparameters In this section, we list the used hyperparameters for our dif- ferent experiments (both quantitative and qualitative) of the article

    Experiments 7.1. Hyperparameters In this section, we list the used hyperparameters for our dif- ferent experiments (both quantitative and qualitative) of the article. Base Nerfacto Model Configuration. We use most of the default Nerfstudio setup including with 16 hash grids an...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.