Pith. sign in

REVIEW 5 major objections 5 minor 55 references

PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-Global Curriculum

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Open-vocabulary 3D semantic segmentation reaches 59.5% mIoU on ScanNet without manual annotations using a two-stage partial-to-global curriculum.

desk verdict PGOV3D looks like a genuine incremental win for annotation-free 3D segmentation, but the headline number depends on an unstated train/eval split and a self-training loop that needs explicit leakage control. read the letter →

arxiv 2506.23607 v1 pith:4FCTIMDU submitted 2025-06-30 cs.CV

Shiqi Zhang , Sha Zhang , Jiajun Deng , Yedong Shen , Mingxiao MA , Yanyong Zhang This is my paper · ORCID
classification cs.CV
keywords open-vocabulary3Dsemanticsegmentationpartial-to-globalcurriculuminter-frameconsistencypseudo-labelgenerationmulti-modallargelanguagemodelzero-shotdomaintransferRGB-DpointcloudScanNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-vocabulary 3D semantic segmentation, labeling every point in a scene with categories never seen during training, currently borrows text-aligned features from multi-view images by projecting them onto 3D points. This paper argues that such global-only training throws away the dense semantic content of the images themselves. PGOV3D instead pre-trains a 3D network on partial point clouds projected from individual RGB-D frames, using pixel-wise pseudo labels produced by a multimodal large language model and a 2D segmentation foundation model, while an inter-frame consistency loss stabilizes features across views. It then fine-tunes on complete scene point clouds using high-confidence pseudo labels generated by the pre-trained model from the aggregated scene vocabulary. The paper reports 59.5% mIoU and 73.2% mAcc on ScanNet, 9.3% mIoU on ScanNet200, and 43.2% mIoU transferring from ScanNet to S3DIS without fine-tuning.

What carries the argument

The carrying mechanism is the partial-to-global curriculum, implemented as two training stages. In stage one, RGB-D frames are converted to partial point clouds by pixel-wise depth projection; a multimodal large language model prompted with chain-of-thought generates an object vocabulary per image, a 2D segmentation foundation model grounds those vocabulary entries as pixel masks, and the sparse-convolution 3D U-Net aligns its point features with CLIP text embeddings under an alignment loss plus a consistency loss that pulls matched physical points from consecutive frames together by cosine similarity. In stage two, the per-image vocabularies are aggregated into a scene vocabulary, the full point cloud is voxelized and repeatedly sampled, and the pre-trained model's averaged per-point predictions, filtered to high confidence, become pseudo labels for fine-tuning the same network.

What would settle it

Re-run the pipeline under an explicitly disjoint scene split: exclude every frame and every scan belonging to evaluation scenes from both pseudo-label generation and fine-tuning, then measure ScanNet mIoU. If the score drops to the level of the strongest baseline, the reported 59.5% would be an artifact of leakage rather than generalization.

Watch

Extended reading notes

Core claim

The central claim is that the gap between 2D image semantics and 3D scene geometry is best crossed by a curriculum: first learn point-text alignment on dense, geometrically simple partial point clouds, where pixel-level pseudo labels from pretrained 2D models give rich supervision; then adapt to sparse, complex full scenes by fine-tuning on point-wise pseudo labels that the pre-trained model itself produces. The auxiliary inter-frame consistency module is what makes this trajectory work, because it forces the same physical point to keep a stable feature across adjacent frames, reducing the semantic noise that view-dependent 2D supervision would otherwise inject. On the paper's evaluation, PGOV3D reaches 59.5% mIoU and 73.2% mAcc on ScanNet, outperforming all annotation-free baselines, and reaches 43.2% mIoU on S3DIS in zero-shot domain transfer.

Load-bearing premise

The weakest load-bearing premise is that the training data, the 25,000 RGB-D frames and the 1,201 scans, are taken only from the ScanNet training split and that no evaluation scene leaks into the pseudo-label generation process, but the paper does not state this split or exclusion.

Editorial extensions

If this is right

  • Unannotated RGB-D video becomes a direct training signal for open-vocabulary 3D segmentation, reducing the need for dense 3D annotation.
  • The inter-frame consistency loss should make point features temporally stable, which matters for robotics and online perception, not just offline benchmarks.
  • Fine-tuning on complete scenes requires no predefined category list, so the procedure can be applied to new indoor datasets as long as their RGB-D frames and scene point clouds are available.
  • The reported base/novel split results suggest that the method's ability to recognize unseen categories improves as the number of novel classes grows, a desirable behavior for open-world deployment.
  • The zero-shot transfer result from ScanNet to S3DIS indicates the learned point-text alignment carries across datasets and label distributions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is the same curriculum on outdoor driving data, where multi-camera RGB-D video supplies dense partial supervision and accumulated LiDAR supplies the global scene point cloud.
  • The recipe generalizes beyond semantic segmentation: pretrain a 3D representation from dense, 2D-supervised partial views, then self-distill onto the full 3D representation, which could be applied to instance or panoptic segmentation.
  • The consistency module is only evaluated at one loss weight and one frame sampling rate; how its benefit scales with viewpoint density would clarify whether it is correcting occlusion noise or mostly acting as a regularizer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. PGOV3D proposes a two-stage training scheme for open-vocabulary 3D semantic segmentation. In stage 1, a U-Net backbone is pre-trained on partial point clouds obtained by un-projecting RGB-D frames, using pseudo-labels produced by LLaVA-NeXT and Grounded-SAM, together with an inter-frame consistency loss that pulls features of matched 3D points closer across views. In stage 2, the model is fine-tuned on complete scene-level point clouds using high-confidence point-wise pseudo-labels generated by the pre-trained model via repeated grid sampling and probability averaging. The paper reports 59.5 mIoU / 73.2 mAcc on ScanNet, 9.3 mIoU on ScanNet200, and 43.2 mIoU on S3DIS in zero-shot domain transfer, surpassing annotation-free baselines. Ablations show consistent gains from the fine-tuning stage (+4.8 mIoU), the consistency module (+1.4 in stage 1), and the use of pre-trained weights in stage 2 (+0.7).

Significance. If the results are reproducible, the paper would be a meaningful step for annotation-free open-vocabulary 3D semantic segmentation, especially the partial-to-global curriculum and the inter-frame consistency regularizer. The ablations are internally consistent and each contribution appears to add value. However, the manuscript leaves several procedural details underspecified, most importantly the dataset split for pre-training and fine-tuning, which is essential for ruling out label leakage. The significance is therefore conditional on the authors clarifying and, if necessary, correcting the evaluation protocol.

major comments (5)
  1. [Sec. 4.1] The evaluation protocol is not completely specified. The text says pre-training uses 25k frames sampled every 100 frames from RGB-D sequences and fine-tuning is conducted on 1,201 reconstructed indoor scans, but it never states that these are restricted to the official ScanNet training split, nor that the evaluation scenes were excluded from the stage-2 pseudo-label generation loop. Since stage 2 fine-tunes on complete point clouds using pseudo-labels generated from those same point clouds, any evaluation scene that is included in training could be observed through its own pseudo-labels, inflating the reported 59.5 mIoU. Similarly, for the S3DIS transfer result, the manuscript says the model is trained on ScanNet and evaluated on Area 5, but it does not explicitly say whether the ScanNet-trained model is frozen, whether any S3DIS data (e.g., other areas) is used for validation, or how the class vocabularies are matched between ScanNet and S3DIS. Please specify the exact train/eval split, the procedure for ensuring no evaluation scene is used in either stage, and the exact evaluation setting for each benchmark.
  2. [Sec. 3.2] The 'Vision-Language Alignment on Full Scenes' paragraph states that the model is fine-tuned by optimizing the cosine similarity loss L_consistency as defined in Eq. 4. However, Eq. 4 is the inter-frame consistency loss that requires matched physical points between two partial views; it is not directly applicable to a single complete point cloud. The ablations in Table 4 and the surrounding text suggest that stage 2 actually uses the alignment loss of Eq. 2. This is a technical error in the method description that must be corrected, and the exact training objective for stage 2 should be stated.
  3. [Sec. 3.1] The inter-frame consistency module relies on a set A of matched physical points across frames (Eqs. 3–4), but the manuscript does not describe how these correspondences are established. Since each RGB-D frame is un-projected into world coordinates via Eq. (1), a natural approach would be spatial nearest neighbor matching, but the distance threshold, the treatment of occlusions and duplicate points, and the choice of frame pairs (consecutive only, or all overlapping) are not specified. This is a load-bearing detail for a proposed module whose ablation shows a non-trivial gain, so it must be presented carefully.
  4. [Sec. 3.2] The pseudo-label generation for the full-scene fine-tuning is described only qualitatively. The manuscript mentions 'high-confidence' point-wise entities and a 'probabilistic smoothing strategy based on repeated grid sampling', but it does not define the confidence threshold, the number of sampling repetitions, or the exact rule for selecting 'reliable point-entity pairs'. These are free parameters that could materially affect the reported improvements. Please provide the exact procedure and, ideally, an ablation showing sensitivity to these choices.
  5. [Sec. 4.2 / Fig. 1] The paper motivates the partial-to-global curriculum by criticizing the 'global-only training paradigm' (Fig. 1a), but it never directly compares against a global-only training baseline under identical conditions. The comparisons in Table 1 include methods such as OpenScene and OV3D, but those use different architectures, different supervision sources, and different training data, so they do not isolate the effect of the curriculum. An ablation that trains the proposed architecture directly on global point clouds from scratch (without the partial-scene pre-training) would strengthen the central claim that the two-stage curriculum, rather than merely the use of image-derived pseudo-labels, is responsible for the observed gains.
minor comments (5)
  1. [Sec. 1, 3.1, 4.2] There are several typos and editorial issues: 'paradgim' (Sec. 1), 'facilate' (Sec. 3.2), 'Austrilia' (author affiliation), 'parse' should be 'partial' in Sec. 3.1, and Table 6's title says 'consistency module' while the content concerns pre-trained weights. These should be fixed.
  2. [Sec. 2.3 and references] The reference list contains two entries labeled 'OV3D' with different meanings: [4] is 'Open-vocabulary 3d object detection' (CVPR 2023), while [49] is 'Open-vocabulary 3d semantic segmentation with foundation models' (CVPR 2024). In Table 1 the compared 'OV3D' is presumably the latter; please disambiguate the citations to avoid confusion.
  3. [Sec. 3.1, Eq. (2)] In Eq. (2), N is not defined; it should be clarified whether N is the number of points with assigned entities, or the total number of points in the partial scene. If unlabeled points are ignored, that should be stated.
  4. [Sec. 4.1] The manuscript uses the term 'zero-shot' for the ScanNet evaluation, but stage 2 fine-tunes on unlabeled ScanNet scans, so the setting is more accurately described as 'annotation-free'. Consider using the latter term consistently to avoid confusion with strict zero-shot evaluation.
  5. [Table 2] In the base/novel experiments, replacing base-category point-level entities with ground-truth textual labels is a peculiar way to simulate base supervision; this means the novel categories may still have been seen as pseudo-labels during stage 1 pre-training. The experimental setup should be described more carefully, and the claim that novel categories are 'unseen' should be qualified.

Circularity Check

0 steps flagged · score 2.0 of 10

No demonstrated circularity; stage-2 pseudo-label self-training is not shown to overlap the evaluation set, and the only self-citation is non-load-bearing.

full rationale

The claimed derivation chain is not circular by construction. Stage 1 (Section 3.1) supervises the 3D network with pixel-wise entities generated by external models (LLaVA-NeXT, Grounded-SAM) and aligns point features to CLIP text embeddings; these are independent of the paper's own outputs. The inter-frame consistency loss (Eq. 4) is an auxiliary regularizer between matched physical points in adjacent frames, not a relabeling of the evaluation metric. Stage 2 (Section 3.2) is a pseudo-label self-training loop: the pre-trained model generates 'high-confidence point-wise entity' labels on global point clouds and the model is fine-tuned on those labels. Self-training is a standard semi-supervised technique and is not circular per se unless the pseudo-labeled scenes overlap the evaluation scenes. Section 4.1 states only that 'fine-tuning is conducted on 1,201 reconstructed indoor scans' and does not explicitly state that these are the ScanNet training split or that evaluation scenes were excluded from pseudo-label generation; this is a reproducibility/protocol gap that could threaten the zero-shot claim, but the text does not demonstrate that the final 59.5 mIoU is equivalent to the fine-tuning input by construction. The only self-citation (ref. [39] in Related Work) is not load-bearing. Score 2 reflects the minor non-load-bearing self-citation; no circular step is established.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities. Its central result is carried by off-the-shelf teachers (LLaVA-NeXT, Grounded-SAM, CLIP), by an unvalidated self-training stage, and by several hand-chosen or unreported hyperparameters.

free parameters (4)
  • lambda = 0.2
    Balancing weight for the inter-frame consistency loss in Eq. 5; set by hand without sensitivity analysis.
  • pretraining frame sampling rate = 25,000 frames (every 100 frames)
    Controls how many partial-scene views the model sees; no ablation is run on this quantity.
  • high-confidence pseudo-label threshold = not reported
    Stage 2 keeps high-confidence point-entity pairs, but the cutoff is never specified.
  • grid sampling repetition count = not reported
    The probabilistic smoothing repeats random voxel sampling; the number of repeats is not given.
assumptions (6)
  • standard math Pinhole camera projection with known intrinsics and poses maps RGB-D pixels to 3D points (Eq. 1).
    Standard projection model used without derivation.
  • domain assumption ScanNet depth values are accurate enough for partial point clouds to be geometrically faithful.
    Section 3.1 uses depth projection directly, although Figure 4 acknowledges depth estimation errors.
  • domain assumption LLaVA-NeXT object lists and Grounded-SAM masks are correct open-vocabulary pixel-wise labels.
    Section 3.1 Pixel-wise Entity Generation treats these outputs as supervision without verification.
  • domain assumption CLIP text embeddings form a valid shared space for aligning 3D point features.
    The alignment loss in Eq. 2 assumes cosine distance in CLIP space measures semantic similarity.
  • domain assumption The ScanNet training split is the only source of pretraining frames and fine-tuning scans.
    Section 4.1 does not state the split, yet zero-shot evaluation depends on excluding evaluation scenes.
  • ad hoc to paper High-confidence self-generated pseudo-labels in stage 2 are reliable enough to improve the model.
    Section 3.2 relies on self-training without comparing pseudo-labels to ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-Global Curriculum." pith.science (2026). https://pith.science/paper/4FCTIMDU

@misc{pith2026250623607,
  author       = {Pith},
  title        = {Pith review of: PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-Global Curriculum},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4FCTIMDU}},
  note         = {Machine review of arXiv:2506.23607}
}
read the original abstract

Existing open-vocabulary 3D semantic segmentation methods typically supervise 3D segmentation models by merging text-aligned features (e.g., CLIP) extracted from multi-view images onto 3D points. However, such approaches treat multi-view images merely as intermediaries for transferring open-vocabulary information, overlooking their rich semantic content and cross-view correspondences, which limits model effectiveness. To address this, we propose PGOV3D, a novel framework that introduces a Partial-to-Global curriculum for improving open-vocabulary 3D semantic segmentation. The key innovation lies in a two-stage training strategy. In the first stage, we pre-train the model on partial scenes that provide dense semantic information but relatively simple geometry. These partial point clouds are derived from multi-view RGB-D inputs via pixel-wise depth projection. To enable open-vocabulary learning, we leverage a multi-modal large language model (MLLM) and a 2D segmentation foundation model to generate open-vocabulary labels for each viewpoint, offering rich and aligned supervision. An auxiliary inter-frame consistency module is introduced to enforce feature consistency across varying viewpoints and enhance spatial understanding. In the second stage, we fine-tune the model on complete scene-level point clouds, which are sparser and structurally more complex. We aggregate the partial vocabularies associated with each scene and generate pseudo labels using the pre-trained model, effectively bridging the semantic gap between dense partial observations and large-scale 3D environments. Extensive experiments on ScanNet, ScanNet200, and S3DIS benchmarks demonstrate that PGOV3D achieves competitive performance in open-vocabulary 3D semantic segmentation.

Figures

Figures reproduced from arXiv: 2506.23607 by the authors.

Figure 1
Figure 1. Global-Only Training VS Our Partial-to-Global Cur [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of our PGOV3D pipeline. Our framework consists of two stages: In the pre-training stage, the 3D segmentation model are pre-trained on partial scenes derived from multi-view RGB-D images via pixel-wise depth projection. The pixel-wise entity is generated by LLaVA-NeXT [7] and 2D foundation model and serve as aligned supervision signals. An auxiliary inter-frame consistency module is introduced to enforce … view at source ↗
Figure 3
Figure 3. An Illustrative Example of LLaVA-NeXT [7] In￾teraction. By prompting the LLaVA-NeXT [7] with Chain-of￾Thought [42] reasoning, we guide it to generate accurate object￾centric vocabulary for visual grounding. 3D point clouds P𝑖 by leveraging the depth information to establish the relationship between points and text. For each RGB-D image 𝐼𝑖 , given the camera intrinsic matrix K ∈ R 3×3 and pose Ti ∈ R 3×3 , each pixel… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Illustration of domain gap between RGB-D partial [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative segmentation comparison between PGOV3D and OpenScene on the ScanNet dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative segmentation with and without further [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 50 canonical work pages

  1. [1]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision, ” inInternational conference on machine learning . PmLR, 2021, pp. 8748–8763

    work page 2021

  2. [2]

    Openscene: 3d scene understanding with open vocabularies,

    S. Peng, K. Genova, C. Jiang, A. Tagliasacchi, M. Pollefeys, T. Funkhouser et al., “Openscene: 3d scene understanding with open vocabularies, ” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 815–824

    work page 2023

  3. [3]

    3d open-vocabulary panoptic segmentation with 2d-3d vision-language distillation,

    Z. Xiao, L. Jing, S. Wu, A. Z. Zhu, J. Ji, C. M. Jiang, W.-C. Hung, T. Funkhouser, W. Kuo, A. Angelova et al., “3d open-vocabulary panoptic segmentation with 2d-3d vision-language distillation, ” inEuropean Conference on Computer Vision . Springer, 2024, pp. 21–38

    work page 2024

  4. [4]

    Ov3d: Open-vocabulary 3d object detection,

    J. Yang, X. Li, Y. Liu, J. Gu, W. Zhang, and C. Liu, “Ov3d: Open-vocabulary 3d object detection, ” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

    work page 2023

  5. [5]

    Language-driven open- vocabulary 3d scene understanding,

    R. Ding, J. Yang, C. Xue, W. Zhang, S. Bai, and X. Qi, “Language-driven open- vocabulary 3d scene understanding, ” Nov 2022

    work page 2022

  6. [6]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Y. Li, and Y. J. Lee, “Improved baselines with visual instruction tuning, ” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 296–26 306

    work page 2024

  7. [7]

    Llava-next: Improved reasoning, ocr, and world knowledge,

    H. Liu, C. Li, Y. Li, B. Li, Y. Zhang, S. Shen, and Y. J. Lee, “Llava-next: Improved reasoning, ocr, and world knowledge, ” January 2024. [Online]. Available: https://llava-vl.github.io/blog/2024-01-30-llava-next/

    work page 2024

  8. [8]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. White- head, A. C. Berg, W.-Y. Loet al., “Segment anything, ” inProceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 4015–4026

    work page 2023

Show all 55 references
  1. [9]

    Segment everything everywhere all at once,

    X. Zou, J. Yang, H. Zhang, F. Li, L. Li, J. Wang, L. Wang, J. Gao, and Y. J. Lee, “Segment everything everywhere all at once, ”Advances in neural information processing systems, vol. 36, pp. 19 769–19 782, 2023

    2023

  2. [10]

    Lerf: Language embedded radiance fields,

    J. Kerr, C. M. Kim, K. Goldberg, A. Kanazawa, and M. Tancik, “Lerf: Language embedded radiance fields, ” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 19 729–19 739

    2023

  3. [11]

    Langsplat: 3d language gaussian splatting,

    M. Qin, W. Li, J. Zhou, H. Wang, and H. Pfister, “Langsplat: 3d language gaussian splatting, ” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 051–20 060

    2024

  4. [12]

    Diff3detr: Agent-based diffusion model for semi- supervised 3d object detection,

    J. Deng, J. Lu, and T. Zhang, “Diff3detr: Agent-based diffusion model for semi- supervised 3d object detection, ” in European Conference on Computer Vision . Springer, 2024, pp. 57–73

    2024

  5. [13]

    Oneformer3d: One transformer for unified point cloud segmentation,

    M. Kolodiazhnyi, A. Vorontsova, A. Konushin, and D. Rukhovich, “Oneformer3d: One transformer for unified point cloud segmentation, ” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 20 943– 20 953

    2024

  6. [14]

    Voxelnet: End-to-end learning for point cloud based 3d object detection,

    Y. Zhou and O. Tuzel, “Voxelnet: End-to-end learning for point cloud based 3d object detection, ” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4490–4499

    2018

  7. [15]

    Point transformer v3: Simpler faster stronger,

    X. Wu, L. Jiang, P.-S. Wang, Z. Liu, X. Liu, Y. Qiao, W. Ouyang, T. He, and H. Zhao, “Point transformer v3: Simpler faster stronger, ” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 4840–4851

    2024

  8. [16]

    Point transformer,

    H. Zhao, L. Jiang, J. Jia, P. H. Torr, and V. Koltun, “Point transformer, ” inProceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 16 259– 16 268

    2021

  9. [17]

    Pointgroup: Dual-set point grouping for 3d instance segmentation,

    L. Jiang, H. Zhao, S. Shi, S. Liu, C.-W. Fu, and J. Jia, “Pointgroup: Dual-set point grouping for 3d instance segmentation, ” inProceedings of the IEEE/CVF conference on computer vision and Pattern recognition , 2020, pp. 4867–4876

    2020

  10. [18]

    Mask-attention-free trans- former for 3d instance segmentation,

    X. Lai, Y. Yuan, R. Chu, Y. Chen, H. Hu, and J. Jia, “Mask-attention-free trans- former for 3d instance segmentation, ” inProceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 3693–3703

    2023

  11. [19]

    Query refinement transformer for 3d instance segmentation,

    J. Lu, J. Deng, C. Wang, J. He, and T. Zhang, “Query refinement transformer for 3d instance segmentation, ” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 18 516–18 526

    2023

  12. [20]

    Superpoint transformer for 3d scene instance segmentation,

    J. Sun, C. Qing, J. Tan, and X. Xu, “Superpoint transformer for 3d scene instance segmentation, ” inProceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 2, 2023, pp. 2393–2401

    2023

  13. [21]

    Softgroup for 3d instance segmentation on point clouds,

    T. Vu, K. Kim, T. M. Luu, T. Nguyen, and C. D. Yoo, “Softgroup for 3d instance segmentation on point clouds, ” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 2708–2717

    2022

  14. [22]

    Ins-conv: Incremental sparse con- volution for online 3d segmentation,

    L. Liu, T. Zheng, Y.-J. Lin, K. Ni, and L. Fang, “Ins-conv: Incremental sparse con- volution for online 3d segmentation, ” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 18 975–18 984

    2022

  15. [23]

    Fusion-aware point convolution for online semantic 3d scene segmentation,

    J. Zhang, C. Zhu, L. Zheng, and K. Xu, “Fusion-aware point convolution for online semantic 3d scene segmentation, ” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 4534–4543

    2020

  16. [24]

    Learning multi-view aggregation in the wild for large-scale 3d semantic segmentation,

    D. Robert, B. Vallet, and L. Landrieu, “Learning multi-view aggregation in the wild for large-scale 3d semantic segmentation, ” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 5575–5584

    2022

  17. [25]

    Mask3d: Mask transformer for 3d semantic instance segmentation,

    J. Schult, F. Engelmann, A. Hermans, O. Litany, S. Tang, and B. Leibe, “Mask3d: Mask transformer for 3d semantic instance segmentation, ” in2023 IEEE Interna- tional Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 8216–8223

    2023

  18. [26]

    Clip2scene: Towards label-efficient 3d scene understanding by clip,

    R. Chen, Y. Liu, L. Kong, X. Zhu, Y. Ma, Y. Li, Y. Hou, Y. Qiao, and W. Wang, “Clip2scene: Towards label-efficient 3d scene understanding by clip, ” pp. 7020– 7030, 2023

    2023

  19. [27]

    Yolo-world: Real-time open-vocabulary object detection,

    T. Cheng, L. Song, Y. Ge, W. Liu, X. Wang, and Y. Shan, “Yolo-world: Real-time open-vocabulary object detection, ” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 16 901–16 911

    2024

  20. [28]

    Pla: Language-driven open-vocabulary 3d scene understanding,

    R. Ding, J. Yang, C. Xue, W. Zhang, S. Bai, and X. Qi, “Pla: Language-driven open-vocabulary 3d scene understanding, ” inIEEE Conference on Computer Vision and Pattern Recognition, 2023, pp. 7010–7019

    2023

  21. [29]

    Language-driven semantic segmentation,

    B. Li, K. Q. Weinberger, S. Belongie, V. Koltun, and R. Ranftl, “Language-driven semantic segmentation, ”arXiv preprint arXiv:2201.03546, 2022

    2022 arXiv

  22. [30]

    Open-vocabulary semantic segmentation with mask-adapted clip,

    F. Liang, B. Wu, X. Dai, K. Li, Y. Zhao, H. Zhang, P. Zhang, P. Vajda, and D. Mar- culescu, “Open-vocabulary semantic segmentation with mask-adapted clip, ” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2023, pp. 7061–7070

    2023

  23. [31]

    Openseene: 3d scene understanding with open vocabularies,

    S. Peng, K. Genova, C. Jiang, A. Tagliasacchi, M. Pollefeys, T. Funkhouser et al., “Openseene: 3d scene understanding with open vocabularies, ” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 815–824

    2023

  24. [32]

    Scaling open-vocabulary image segmen- tation with image-level labels,

    G. Ghiasi, X. Gu, Y. Cui, and T.-Y. Lin, “Scaling open-vocabulary image segmen- tation with image-level labels, ” European Conference on Computer Vision , pp. 540–557, 2022

    2022

  25. [33]

    Semantic gaussians: Open-vocabulary scene understanding with 3d gaussian splatting,

    J. Guo, X. Ma, Y. Fan, H. Liu, and Q. Li, “Semantic gaussians: Open-vocabulary scene understanding with 3d gaussian splatting, ”arXiv preprint arXiv:2403.15624, 2024

    2024 arXiv

  26. [34]

    Side adapter network for open- vocabulary semantic segmentation,

    M. Xu, Z. Zhang, F. Wei, H. Hu, and X. Bai, “Side adapter network for open- vocabulary semantic segmentation, ”Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 2945–2954, 2023

    2023

  27. [35]

    Openins3d: Snap and lookup for 3d open-vocabulary instance segmentation,

    Q. Meng, J. Yang, X. Li, Y. Liu, J. Gu, W. Zhang, and C. Liu, “Openins3d: Snap and lookup for 3d open-vocabulary instance segmentation, ” inProceedings of the IEEE/CVF International Conference on Computer Vision , 2023

    2023

  28. [36]

    Open vocabulary 3d scene understanding via geometry guided self-distillation,

    P. Wang, Y. Wang, S. Li, Z. Zhang, Z. Lei, and L. Zhang, “Open vocabulary 3d scene understanding via geometry guided self-distillation, ” inEuropean Conference on Computer Vision, 2024, pp. 442–460

    2024

  29. [37]

    Open-vocabulary 3d semantic segmentation with text-to-image diffusion models,

    X. Zhu, H. Zhou, P. Xing, L. Zhao, H. Xu, J. Liang, A. Hauptmann, T. Liu, and A. Gallagher, “Open-vocabulary 3d semantic segmentation with text-to-image diffusion models, ” inEuropean Conference on Computer Vision , 2024, pp. 357–375

    2024

  30. [38]

    Gaussian grouping: Segment and edit anything in 3d scenes,

    M. Ye, M. Danelljan, F. Yu, and L. Ke, “Gaussian grouping: Segment and edit anything in 3d scenes, ”European Conference on Computer Vision , pp. 162–179, 2024

    2024

  31. [39]

    Og- gaussian: Occupancy based street gaussians for autonomous driving,

    Y. Shen, X. Zhang, Y. Duan, S. Zhang, H. Li, Y. Wu, J. Ji, and Y. Zhang, “Og- gaussian: Occupancy based street gaussians for autonomous driving, ” arXiv preprint arXiv:2502.14235, 2025

    2025 arXiv

  32. [40]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis, ” Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021

    2021

  33. [41]

    3d gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering, ”ACM Transactions on Graphics, vol. 42, no. 4, pp. 1–14, 2023

    2023

  34. [42]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., “Chain-of-thought prompting elicits reasoning in large language models, ” Advances in neural information processing systems, vol. 35, pp. 24 824–24 837, 2022

    2022

  35. [43]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y. Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang, “Grounded sam: Assembling open-world models for diverse visual tasks, ” 2024

    2024

  36. [44]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes,

    A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes, ” inProc. Computer Vision and Pattern Recognition (CVPR), IEEE , 2017

    2017

  37. [45]

    Language-grounded indoor 3d semantic segmentation in the wild,

    D. Rozenberszki, O. Litany, and A. Dai, “Language-grounded indoor 3d semantic segmentation in the wild, ” inProceedings of the European Conference on Computer Vision (ECCV), 2022

    2022

  38. [46]

    3d semantic parsing of large-scale indoor spaces,

    I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese, “3d semantic parsing of large-scale indoor spaces, ” in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016, pp. 1534–1543

    2016

  39. [47]

    Mseg: A composite dataset for multi-domain semantic segmentation,

    J. Lambert, Z. Liu, O. Sener, J. Hays, and V. Koltun, “Mseg: A composite dataset for multi-domain semantic segmentation, ” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 2879–2888

    2020

  40. [48]

    Regionplc: Regional point- language contrastive learning for open-world 3d scene understanding,

    J. Yang, R. Ding, W. Deng, Z. Wang, and X. Qi, “Regionplc: Regional point- language contrastive learning for open-world 3d scene understanding, ” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 19 823–19 832

    2024

  41. [49]

    Open-vocabulary 3d semantic segmentation with foundation models,

    L. Jiang, S. Shi, and B. Schiele, “Open-vocabulary 3d semantic segmentation with foundation models, ” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 21 284–21 294. PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-...

    2024

  42. [50]

    4d spatio-temporal convnets: Minkowski convolutional neural networks,

    C. Choy, J. Gwak, and S. Savarese, “4d spatio-temporal convnets: Minkowski convolutional neural networks, ” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , Jun 2019. [Online]. Available: http: //dx.doi.org/10.1109/cvpr.2019.00319

    2019

  43. [51]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization, ” Learn- ing,Learning, Nov 2017

    2017

  44. [52]

    Generative zero-shot learning for semantic segmentation of 3d point clouds,

    B. Michele, A. Boulch, G. Puy, M. Bucher, and R. Marlet, “Generative zero-shot learning for semantic segmentation of 3d point clouds, ” in 2021 International Conference on 3D Vision (3DV) . IEEE, 2021, pp. 992–1002

    2021

  45. [53]

    Transductive zero- shot learning for 3d point cloud classification,

    A. Cheraghian, S. Rahman, D. Campbell, and L. Petersson, “Transductive zero- shot learning for 3d point cloud classification, ” inProceedings of the IEEE/CVF winter conference on applications of computer vision , 2020, pp. 923–933

    2020

  46. [54]

    L-seg: An end-to-end unified framework for multi-lesion segmentation of fundus images,

    S. Guo, T. Li, H. Kang, N. Li, Y. Zhang, and K. Wang, “L-seg: An end-to-end unified framework for multi-lesion segmentation of fundus images, ”Neurocomputing, vol. 349, pp. 52–63, 2019

    2019

  47. [55]

    Xmask3d: Cross-modal mask reasoning for open vocabulary 3d semantic segmentation,

    Z. Wang, Y. Wang, X. Yu, J. Zhou, and J. Lu, “Xmask3d: Cross-modal mask reasoning for open vocabulary 3d semantic segmentation, ”Advances in Neural Information Processing Systems, vol. 37, pp. 74 922–74 944, 2024

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.