Pith. sign in

REVIEW 4 major objections 5 minor 26 references

SaaF: Scene-Specific Ambiguity-Aware 3D Language Fields towards Interactive Real-World Object Retrieval

T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SaaF, a 3D language field built on Gaussian Splatting, estimates whether a user's object query is ambiguous from the L2 norm of the compressed text feature, asks for clarification when it is, and retrieves the target otherwise.

desk verdict Real retrieval gains, but the ambiguity-aware training as written has no text-feature loss — the novelty hinges on a mechanism the equations don't describe. read the letter →

arxiv 2607.16309 v1 pith:BH4C3TI5 submitted 2026-07-14 cs.CV

Yuga Yano , Daiju Kanaoka , Hakaru Tamukoh , Yasutomo Kawanishi This is my paper · ORCID
classification cs.CV
keywords 3DlanguagefieldsGaussianSplattingopen-vocabularyobjectretrievalambiguitydetectionmetriclearningCLIPfeaturesinteractiveroboticsnaturalgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a scene-specific 3D language field can do two things at once: retrieve objects from open-vocabulary text queries and detect when the query is ambiguous, so that a robot can ask for clarification instead of guessing wrong. The proposed method, SaaF, learns a compressed feature space where ambiguous text descriptions land near the origin (small vector length) and specific ones have larger length, making the L2 norm a readout for ambiguity. If true, service robots operating in cluttered real scenes could handle vague instructions like 'bring the bottle' by asking 'which one?' rather than picking an arbitrary candidate. The paper reports improved retrieval accuracy over previous 3D language fields and demonstrates that ambiguous versus specific queries separate cleanly under a per-scene threshold.

What carries the argument

The L2-norm ambiguity score: after a text query is compressed by the trained encoder, its vector length is compared with a per-scene threshold mu; a low norm signals ambiguity and triggers a clarifying question, while a high norm permits standard cosine-similarity retrieval. The feature space is shaped by a metric-learning autoencoder trained with a large-margin cosine loss (pulling same-instance image features together, pushing different instances apart) plus a reconstruction loss, which the paper claims forces ambiguous descriptors to cluster near the origin and unique descriptors to have larger norms. Additional components are best-frame selection (embedding the frame with the largest tra

What would settle it

A concrete check: train the autoencoder exactly as in the paper but with the ambiguous text-label generation removed (the paper's own ablation condition), then compress held-out ambiguous and specific text queries for a scene and compare their L2 norms. If the norms still separate cleanly, the paper's claim that multi-level text labels are needed for ambiguity awareness is falsified; if they do not separate, then the norm threshold relies on those labels, and its robustness can be probed by testing the per-scene mu on a second scene without re-tuning.

Watch

Extended reading notes

Core claim

SaaF constructs a 3D language field by embedding compressed CLIP features into Gaussian Splatting primitives, and trains a metric-learning autoencoder with a large-margin cosine loss plus reconstruction loss. The paper argues that this training pulls features of the same object instance together, separates different instances, and concentrates ambiguous text labels (e.g., supercategories that cover multiple objects) near the origin while pushing unique descriptions to larger norms. Consequently, the L2 norm of a compressed user-query feature becomes an ambiguity score: if it falls below a manually set threshold mu, the system classifies the query as ambiguous and requests more information; o

Load-bearing premise

The load-bearing premise is that the L2 norm of a compressed text query is a reliable ambiguity signal, yet the training loss in Eqs. (1)-(7) contains no text-feature term, and the threshold mu is set manually per scene; if text features are not actually shaped by the objective or the norm does not transfer across scenes without hand-tuning, the ambiguity-detection claim collapses.

Editorial extensions

If this is right

  • Service robots can decide between acting and asking for clarification based on the norm of a compressed text query, reducing wrong retrievals in scenes with visually similar objects.
  • Retrieval accuracy improves over prior 3D language fields, especially for objects sharing the same supercategory, as shown by higher IoU on 3D-OVS and LERF scenes.
  • Open-vocabulary queries are handled without per-query training, and the ambiguity check adds little cost because it only compresses the query rather than decoding all pixels.
  • The method depends on the quality of the underlying segmentation tracker; wrong instance IDs can collapse features toward the origin and degrade ambiguity detection.
  • A per-scene threshold mu must be set manually, and automatic threshold determination is left as future work, limiting immediate deployment in new environments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The norm-based ambiguity signal might be obtainable from image-feature training alone; the paper's ablation shows that removing text labels degrades accuracy, so a direct test of whether text labels are strictly necessary for norm separation would clarify the mechanism.
  • The manual threshold mu could be learned or calibrated on a validation set of labeled queries, which would remove the named limitation and make the method more plug-and-play across scenes.
  • The same feature-space design may extend to hierarchical object understanding (whole/part/sub-part) or to queries that intentionally match multiple objects, where the system could list all candidates instead of asking for clarification.
  • Since tracking errors directly hurt ambiguity detection, combining this approach with camera-parameter-aware tracking could improve robustness in real-world robot deployments.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SaaF, a Gaussian Splatting-based 3D language field that combines a metric-learning autoencoder with VLM-generated multi-level text labels to improve object retrieval and to detect ambiguous text queries. The method first collects per-instance tracked object masks with SAM2, generates text labels of varying specificity via GPT-4o, trains an autoencoder with a large-margin cosine loss and reconstruction loss, and embeds the compressed CLIP features into 3D Gaussians. At inference, a query is classified as ambiguous if the L2 norm of its compressed text feature falls below a hand-set threshold μ; otherwise, pixels are retrieved by cosine similarity above τ. Experiments on 3D-OVS and the LERF Figurines scene report higher retrieval IoU than Lang-SAM, LERF, and LangSplat, and qualitative evidence that ambiguous queries have lower compressed-feature norms. The central ambiguity-aware training claim, however, is not supported by the equations in Sec. III-B, which contain no text-feature term.

Significance. If the central claims held, SaaF would be a practical step toward interactive open-vocabulary object retrieval, with concrete IoU improvements over strong baselines (e.g., 96.4 vs. 93.5 overall on 3D-OVS, and 46.6 vs. 44.7 on Figurines). The paper also makes an honest attempt at ablation of the proposed components. However, the manuscript's central novelty — learning a feature space that is ambiguity-aware by jointly training on text labels — is not actually demonstrated: the stated training objective (Eqs. 1–7) operates only on image features, and the ambiguity classifier (Eq. 8) relies on a manually tuned threshold. The retrieval accuracy gains may still stand on their own, but the ambiguity-detection claim, which is a headline contribution, currently lacks a clear mechanism and rigorous evaluation.

major comments (4)
  1. [Sec. III-B, Eqs. (1)–(7)] The training objective contains no text-feature term. L_lmc (Eqs. 2–6) is defined entirely over image feature sets E_i and E_j, and L_reconst (Eq. 7) reconstructs CLIP image features F_i. The text labels generated in Sec. III-A.2 never appear in the loss. The paper claims these labels 'teach the model the semantic relationships between ambiguous and specific features,' but no mechanism for text influence on the learned space is specified. Table III shows that text-label type changes mIoU, but the implementation of 'Specific' and 'Multiple' text-label training is not described. This is a load-bearing gap: the ambiguity-aware property is attributed to text-label training, yet the loss cannot be shown to use those labels. Please either add the text-feature loss and training procedure, or revise the claim to state that text labels only provide supervision for the VLM generation and not for t
  2. [Sec. III-C, Eq. (8) and Sec. IV-E] The ambiguity classifier is a threshold on the L2 norm of the compressed text feature, with μ set manually per scene (admitted in Sec. IV-E: 'the threshold μ must be determined manually based on the environment'). The evaluation in Sec. IV-B reports separation using thresholds (0.5 for LERF, and values implied for 3D-OVS) that appear tuned to the same scenes and queries used in the qualitative evaluation. No held-out validation or principled selection of μ is provided. Because the norm distribution depends on the arbitrary scaling of the encoder and on query phrasing, this does not demonstrate robust ambiguity detection. Please report a systematic evaluation with a pre-specified threshold or a validation-set selection, and provide error bars or confusion matrices.
  3. [Sec. III-B, Fig. 3 and 'Feature space'] The proposed mechanism for norm-based ambiguity is not derived from the loss. The text argues that ambiguous features 'are concentrated near the origin' because reconstruction loss encourages Euclidean closeness while L_lmc maintains directional separation. However, Eqs. (1)–(7) only enforce cosine similarity between same-instance features and a margin for different instances; there is no term that ties text features to image features or that explicitly locates ambiguous concepts near the origin. The claimed behavior is plausible in principle but is not a mathematical consequence of the stated objective. Please either provide a derivation or a controlled experiment (e.g., train with and without text labels and measure the norm distribution on held-out ambiguous/specific queries) to support the mechanism.
  4. [Sec. IV-D, Table III] The ablation study is the only evidence that text labels affect training, but the 'Specific' and 'Multiple' conditions are under-specified. It is unclear how text labels are incorporated into the autoencoder training given that the loss (Eq. 1) has no text term. For example, are text features passed through the same encoder, are they used as additional training samples, or are they only used to select image features? Without this information, the large mIoU differences (50.4 to 87.9) cannot be interpreted. Please clarify the exact training procedure for each row.
minor comments (5)
  1. [Sec. IV-A, Table I] There is a typo in the LERF row: '46,6' should be '46.6'. Please also ensure all numbers use consistent decimal notation.
  2. [Sec. IV-B, Figs. 5 and 6] The figures show 'L2 norm' values but do not provide a legend for all bars, and the numeric values listed above Fig. 6 ('3.89 2.82 9.31 1.98 1.91 5.49 2.81') are not labeled with their corresponding queries. Please add axis labels, a legend, and clear annotations.
  3. [Sec. III-A.2] The prompt says 'make about 100 labels for each object,' but the system prompt shown earlier says 'Please output various names that accurately expresses the characteristics.' The number of labels actually generated and the randomness of frame selection are not quantified. Please report the number of labels per object and the prompt used for the 'specific' condition in the ablation.
  4. [Sec. IV-B] The ambiguity evaluation is qualitative only. The paper reports threshold values and individual query norms but no aggregate statistics (e.g., mean and variance for ambiguous vs. specific queries, ROC curve, or accuracy of the detector). Please consider reporting quantitative ambiguity-detection performance.
  5. [Sec. IV-B, bench scene] The text states 'the query fruit scores 0.53, since only green grape is applicable in this scene.' But the figure suggests both 'tart' and 'green grape' are present. Please clarify whether 'fruit' is considered ambiguous or specific in this scene, and how the threshold separates them.

Circularity Check

1 steps flagged · score 6.0 of 10

Ambiguity detection in Eq. (8) reduces to a manually fitted per-scene threshold on a text-feature norm, making the central ambiguity claim partially circular.

  1. fitted input called prediction [Sec. III-C Eq. (8); Sec. IV-B Ambiguity Scores; Sec. IV-E Limitation]
    "The L2 norm of the compressed text feature qcomp is computed; if it falls below a certain threshold µ, the query is classified as ambiguous ... Another practical limitation is that the threshold µ must be determined manually based on the environment."

    The ambiguity decision rule (Eq. 8) depends on μ, a per-scene threshold with no specified selection procedure. In Sec. IV-B, the authors demonstrate separation by setting a threshold (e.g., 0.5 for LERF) on the very queries used for evaluation, and Sec. IV-E admits μ is manually determined per environment. Thus the reported 'robust detection' claim is not a pre-specified, out-of-sample prediction but a decision rule fitted to the test queries. The core ambiguity-detection result reduces to: under a threshold chosen for these scenes, the norm separates these queries.

full rationale

The object-retrieval component (IoU in Tables I-II) is externally benchmarked against LangSplat, LERF, and Lang-SAM on standard datasets, so that part is not circular. However, the novelty claim—ambiguity-aware detection and interaction—rests entirely on Eq. (8), where the L2 norm of the compressed text feature is thresholded by μ. The paper concedes μ is manually set per environment, and the evaluation in Sec. IV-B reports separation using such a threshold on the same queries that are being classified. This makes the central ambiguity-detection claim partially circular: the threshold is a fitted input rather than a learned, held-out decision rule. In addition, the abstract and ablation attribute the effect to training on VLM-generated text labels, but the stated loss (Eqs. 1-7) contains no text-feature term—only image-feature metric and reconstruction losses. This is a serious missing support that compounds the circularity, though by itself it is a correctness gap rather than a reduction. No load-bearing self-citations or uniqueness-import arguments are present. Score 6 reflects that the ambiguity 'prediction' reduces by construction to a manually fitted threshold, while the retrieval accuracy retains independent content.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical entities. The free parameters are dominated by the manually chosen ambiguity threshold μ and unstated margins/weights; the key heuristic (L2-norm ambiguity) is asserted rather than derived. The most serious ledger item is the unstated text-feature loss, which is necessary for the central claim but missing from the paper's equations.

free parameters (4)
  • λ (balance parameter) = not reported
    Balance between metric loss and reconstruction loss in Eq. (1); value is not given and affects the feature space geometry.
  • Margin M = not reported
    Margin in the negative cosine-similarity loss, Eq. (4); not specified in the paper.
  • Ambiguity threshold μ = manually set, e.g., 0.5 for LERF
    Load-bearing for the ambiguity classifier in Eq. (8); the authors acknowledge manual determination is required and future work is needed to automate it.
  • Retrieval threshold τ = not reported
    Pixel similarity threshold in Eq. (9); value is not given, though it affects segmentation output.
assumptions (5)
  • domain assumption CLIP features are a sufficient semantic backbone for open-vocabulary retrieval after compression.
    The method inherits CLIP's image-text alignment and does not verify its failure modes for visually similar scene-specific objects.
  • domain assumption SAM2 multi-object tracking provides consistent instance IDs across frames.
    Tracked masks define the instance labels for metric learning; the authors acknowledge in Sec IV-E that retrieval accuracy depends on tracking accuracy.
  • domain assumption GPT-4o-generated text labels accurately capture object identity and supercategories.
    VLM labels are used without validation; prompts in Sec III-A.2 rely on a black-box model with unspecified version and sampling.
  • ad hoc to paper The L2 norm of a compressed text feature indicates query ambiguity.
    This is the core heuristic in Eq. (8); no derivation or generalization study is provided, and the threshold is manually set.
  • ad hoc to paper Text labels actually enter the training objective.
    The abstract and Sec III-B claim training with text labels, but Eqs. (1)-(7) define the loss using only image features; the mechanism is absent from the written derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SaaF: Scene-Specific Ambiguity-Aware 3D Language Fields towards Interactive Real-World Object Retrieval." pith.science (2026). https://pith.science/paper/BH4C3TI5

@misc{pith2026260716309,
  author       = {Pith},
  title        = {Pith review of: SaaF: Scene-Specific Ambiguity-Aware 3D Language Fields towards Interactive Real-World Object Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BH4C3TI5}},
  note         = {Machine review of arXiv:2607.16309}
}
read the original abstract

We propose Scene-specific Ambiguity-aware 3D Language Fields (SaaF), a novel Gaussian Splatting-based 3D language field designed for interactive object retrieval in a given real-world scene. Interactive object retrieval using natural language is a crucial capability for service robots operating in complex real-world environments. While recent 3D language field methods for object retrieval establish associations between rendered pixels and autoencoder-compressed CLIP features, they suffer from two limitations: (1) reduced discriminability among similar objects due to feature compression, and (2) poor handling of ambiguous queries, often resulting in unstable or incorrect retrieval. To address these limitations, SaaF introduces a metric learning strategy to construct a unified feature space that is both instance-discriminative and ambiguity-aware. (i) To enhance instance-level visual discrimination, SaaF employs metric learning that pulls image features from multiple viewpoints of the same object closer together in the feature space. (ii) To establish ambiguity awareness, the model jointly trains on multiple text labels generated by the proposed method from each tracked object image sequence, including ambiguous descriptions, to learn the semantic relationships between ambiguous and specific features in a target scene. This feature space enables fine-grained visual understanding while allowing the system to estimate query ambiguity and interactively request clarification when needed. Experimental results demonstrate that SaaF not only improves retrieval accuracy over previous methods but also robustly detects and handles ambiguity in the user text queries under open-vocabulary settings.

Figures

Figures reproduced from arXiv: 2607.16309 by the authors.

Figure 1
Figure 1. Target scenario: Conventional 3D language fields only focus on retrieving the most relevant object within a scene and do not account for scenarios where multiple possible candidates exist. We propose a novel 3D language field that simultaneously checks query ambiguity and retrieves an object. Our method quantitatively evaluates query ambiguity before retrieving objects and requests additional user instruction when t… view at source ↗
Figure 2
Figure 2. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. L = Llmc + λLreconst, (1) where λ denotes a balance parameter. Using the cosine similarity-based loss Llmc aims to increase cosine similarity between features of the same instance and to reduce the similarity between features of different instances. Given a set of images Ii of the i-th tracked object instance, a set of CLIP features Fi is extracted. Subsequently, these features are embedded in the trained feature sp… view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: Feature space of the SaaF. In the proposed metric learning, unique features such as Coke and Orange juice are mapped to distant positions in the feature space. On the other hand, ambiguous features such as Round bottle and Shaped bottle that span both instances are map…
Figure 4
Figure 4. Figure 4: Qualitative results in 3D-OVS dataset. Segmentation result in the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Comparison of L2 norm in Figurines. TABLE III ABLATION RESULTS OF 3D-OVS DATASETS. Components mIoU Accuracy (%) Best Frame Text-Label 3D-OVS w/o 41.4 ✓ w/o 50.4 ✓ Specific 87.9 Multiple 86.4 ✓ Multiple 96.4 Table III shows the mIoU scores on the 3D-OVS dataset under di…
Figure 5
Figure 5. Figure 5: Comparison of L2 norm and segmentation results in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 3 linked inside Pith

  1. [1]

    RT-2: Vision-language-action models transfer Web knowledge to robotic control,

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid,et al., “RT-2: Vision-language-action models transfer Web knowledge to robotic control,” inProceedings of the 7th Conference on Robot Learning, pp. 2165–2183, 2023

    2023

  2. [2]

    Unified understanding of environment, task, and human for human- robot interaction in real-world environments,

    Y . Yano, A. Mizutani, Y . Fukuda, D. Kanaoka, T. Ono, and H. Tamukoh, “Unified understanding of environment, task, and human for human- robot interaction in real-world environments,” inProceedings of the 33rd IEEE International Conference on Robot and Human Interactive Communication, pp. 224–230, 2024

    2024

  3. [3]

    Robots that ask for help: Uncertainty alignment for large language model planners,

    A. Z. Ren, A. Dixit, A. Bodrova, S. Singh, S. Tu, N. Brown, P. Xu, L. Takayama, F. Xia, J. Varley, Z. Xu, D. Sadigh, A. Zeng, and A. Majumdar, “Robots that ask for help: Uncertainty alignment for large language model planners,” inProceedings of the 7th Annual Conference on Robot Learning, 2023

    2023

  4. [4]

    Do as I can, not as I say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman,et al., “Do as I can, not as I say: Grounding language in robotic affordances,”arXiv preprint arXiv:2204.01691, 2022

    arXiv 2022

  5. [5]

    ScanQA: 3D question answering for spatial scene understanding,

    D. Azuma, T. Miyanishi, S. Kurita, and M. Kawanabe, “ScanQA: 3D question answering for spatial scene understanding,” inProceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19129–19139, 2022

    2022

  6. [6]

    Sim VQA: Exploring simulated environments for visual question answering,

    P. Cascante-Bonilla, H. Wu, L. Wang, R. Feris, and V . Ordonez, “Sim VQA: Exploring simulated environments for visual question answering,” inProceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5056–5066, 2022

    2022

  7. [7]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” inProceedings of the 38th International Conference on Machine Learning(M. Meila and T. Zhang, eds.), vol. 139, pp. 8748–8763, 2021

    2021

  8. [8]

    3D gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3D gaussian splatting for real-time radiance field rendering,”ACM Transactions on Graphics, vol. 42, July 2023

    2023

Show all 26 references
  1. [9]

    LangSplat: 3D language gaussian splatting,

    M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister, “LangSplat: 3D language gaussian splatting,” inProceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20051– 20060, June 2024

    2024

  2. [10]

    Language embedded 3D gaussians for open-vocabulary scene understanding,

    J.-C. Shi, M. Wang, H.-B. Duan, and S.-H. Guan, “Language embedded 3D gaussians for open-vocabulary scene understanding,” inProceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5333–5343, 2024

    2024

  3. [11]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo,et al., “Segment anything,” inProceedings of the 19th IEEE/CVF International Conference on Computer Vision, pp. 4015–4026, 2023

    2023

  4. [12]

    NeRF: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng, “NeRF: Representing scenes as neural radiance fields for view synthesis,”Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021

    2021

  5. [13]

    Gaussian grouping: Segment and edit anything in 3D scenes,

    M. Ye, M. Danelljan, F. Yu, and L. Ke, “Gaussian grouping: Segment and edit anything in 3D scenes,” inComputer Vision — ECCV 2024, pp. 162–179, 2024

    2024

  6. [14]

    LERF: Language embedded radiance fields,

    J. Kerr, C. M. Kim, K. Goldberg, A. Kanazawa, and M. Tancik, “LERF: Language embedded radiance fields,” inProceedings of the 19th IEEE/CVF International Conference on Computer Vision, pp. 19729– 19739, 2023

    2023

  7. [15]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” inProceedings of the 2021 IEEE/CVF International Conference on Computer Vision, pp. 9650–9660, 2021

    2021

  8. [16]

    OpenGaussian: Towards point-level 3D Gaussian-based open vocabulary understanding,

    Y . Wu, J. Meng, H. LI, C. Wu, Y . Shi, X. Cheng, C. Zhao, H. Feng, E. Ding, J. Wang, and J. Zhang, “OpenGaussian: Towards point-level 3D Gaussian-based open vocabulary understanding,” inProceedings of the 38th Annual Conference on Neural Information Processing Systems, 2024

    2024

  9. [17]

    Segment any 3D Gaussians,

    J. Cen, J. Fang, C. Yang, L. Xie, X. Zhang, W. Shen, and Q. Tian, “Segment any 3D Gaussians,”CoRR, vol. abs/2312.00860, 2023

    2023 arXiv

  10. [18]

    Click-gaussian: Interactive segmentation to any 3D Gaussians,

    S. Choi, H. Song, J. Kim, T. Kim, and H. Do, “Click-gaussian: Interactive segmentation to any 3D Gaussians,” inComputer Vision — ECCV 2024, pp. 289–305, Springer, 2025

    2024

  11. [19]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Doll ´ar, and C. Feichtenhofer, “Sam 2: Segment anything in images and videos,”arXiv preprint arXiv:2...

    2024 arXiv

  12. [20]

    Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su,et al., “Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection,” inComputer Vision — ECCV 2024, pp. 38–55, Springer, 2025

    2024

  13. [21]

    Language-driven semantic segmentation,

    B. Li, Q. K. Weinberger, S. Belongie, V . Koltun, and R. Ranftl, “Language-driven semantic segmentation,” inProceedings of the 2022 International Conference on Learning Representations, 2022

    2022

  14. [22]

    InstructBLIP: Towards general-purpose vision-language models with instruction tuning,

    W. Dai, J. Li, D. LI, A. Tiong, J. Zhao, W. Wang, B. Li, P. N. Fung, and S. Hoi, “InstructBLIP: Towards general-purpose vision-language models with instruction tuning,”Advances in Neural Information Processing Systems, vol. 36, pp. 49250–49267, 2023

    2023

  15. [23]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, pp. 34892– 34916, 2023

    2023

  16. [24]

    GPT-4v(ision) system card,

    OpenAI, “GPT-4v(ision) system card,” 2023. https://cdn. openai.com/papers/GPTV_System_Card.pdf

    2023

  17. [25]

    Weakly supervised 3D open-vocabulary segmen- tation,

    K. Liu, F. Zhan, J. Zhang, M. Xu, Y . Yu, A. El Saddik, C. Theobalt, E. Xing, and S. Lu, “Weakly supervised 3D open-vocabulary segmen- tation,”Advances in Neural Information Processing Systems, vol. 36, pp. 53433–53456, 2023

    2023

  18. [26]

    Language-segment anything,

    L. Medeiros, “Language-segment anything,” 2024. https:// github.com/luca-medeiros/lang-segment-anything (accessed 8. Apr. 2025)

    2024

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.