Pith. sign in

REVIEW 4 major objections 3 minor 48 references

ViGiL3D: A Linguistically Diverse Dataset for 3D Visual Grounding

T0 review · 4 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current 3D visual grounding models lose at least 20 accuracy points on linguistically diverse prompts built from the same ScanNet scenes.

desk verdict A genuinely useful diagnostic benchmark for 3DVG; the performance drops are real, but the paper should attribute them to linguistic diversity more cautiously. read the letter →

arxiv 2501.01366 v2 pith:PJSNZJJP submitted 2025-01-02 cs.CV cs.AIcs.CL

Austin T. Wang , ZeMing Gong , Angel X. Chang This is my paper · ORCID
classification cs.CVcs.AIcs.CL
keywords 3Dvisualgroundinglinguisticdiversitydiagnosticdatasetopen-vocabularyScanNetnegationLLM-scaleddatasetsbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

3D visual grounding is the task of picking out one or more objects in a 3D scene from a natural-language sentence, the kind of capability a robot would need to fetch "the box that says Irish Springs" or "the object under the workstation between the two cabinets." The paper argues that existing 3D visual grounding datasets, including the large LLM-scaled ones, cover only a narrow slice of English: targets are usually named by their exact class, the target is usually the first noun phrase, and negations or coarse-grained references are rare. To make this measurable, the authors introduce ViGiL3D, a hand-annotated diagnostic set of 350 prompts on ScanNet and ScanNet++ scenes that deliberately spans 35 linguistic criteria across target references, anchors, attributes, relationships, and patterns such as negation. They evaluate seven open-vocabulary grounding methods and find that every one drops by at least 20 accuracy points relative to its ScanRefer score, with the best method reaching 26.2% accuracy on ground-truth boxes. A reweighting control that matches ScanRefer's pattern distribution improves performance only marginally, leading the authors to conclude that simply adding more training data will not close the gap unless the right distribution of language is captured.

What carries the argument

The load-bearing object is a 35-criterion linguistic taxonomy together with an automated pipeline that turns a prompt into an augmented scene graph. The taxonomy separates the prompt into target, anchors, attributes, and relationships, and assigns each to one of three categories—language diversity, resolution, and understanding of attributes and relationships—and it tracks specific patterns such as negation and unique lexical bigrams. A GPT-4o-based parser extracts the scene graph and a dependency parser supplies token parses; the authors validate the output against 225 manually annotated prompts, reporting average precision 0.86 and recall 0.91 across 28 binary metrics. ViGiL3D itself is the second half of the machinery: a 350-prompt diagnostic set annotated with the same criteria so that model performance can be broken down by subgroup.

What would settle it

Conduct a controlled comparison on a matched sample: take a few hundred ScanRefer-style prompts that have the same scene distribution, zero-/multi-target structure, and object proposals as ViGiL3D but with ViGiL3D's coverage of the 35 linguistic patterns removed. If any of the seven evaluated models recovers ScanRefer-level accuracy on that set, the drop would be shown to come from task format rather than language diversity; the paper's reweighting control is a small step in this direction but uses only 25 patterns and 350 prompts.

Watch

Extended reading notes

Core claim

ViGiL3D claims to demonstrate that current open-vocabulary 3D visual grounding models are not yet proficient at understanding out-of-distribution natural language prompts. The dataset's 350 prompts were written by human annotators to keep each prompt solvable but requiring every constraint to be parsed; this includes zero-target prompts (descriptions that plausibly describe a scene but do not pick out any object), multi-target prompts, generic and coarse-grained references such as "object" or "appliance," negation, ordinal and arrangement relationships, and viewpoint-dependent anchors. On ScanNet scenes, the best model evaluated, PQ3D, reaches 26.2% accuracy with ground-truth boxes and its F1 is 24.4 points lower than on ScanRefer; other methods show larger drops, and no model is consistently strong across the linguistic subgroups. Comparisons on ScanNet++ show that transferring to new scenes is harder for most models, with only ZSVG3D maintaining its score. The authors also compare the language coverage of eight existing datasets with a GPT-4o-based scene-graph parsing pipeline and report that ViGiL3D has the broadest coverage, including a unique-bigram proportion of 0.52 versus values below 0.30 for most prior datasets.

Load-bearing premise

The load-bearing premise is that the 35 hand-selected linguistic metrics and the GPT-4o parsing pipeline capture the dimensions of language that matter for 3D visual grounding, so the measured performance drops are attributable to linguistic diversity rather than to task-specific artifacts such as zero-target prompts, distractor construction, or the particular scene sample.

Editorial extensions

If this is right

  • Open-vocabulary 3D grounding models that score well on ScanRefer cannot be assumed to handle everyday English; their accuracy on ViGiL3D drops by at least 20 points on the same scenes.
  • Training on more data alone will not fix the gap: the reweighting control in Appendix C.2 improves 3D-GRAND only from 0.18 to 0.19 accuracy when the prompt distribution is matched to its training set.
  • No existing model solves all linguistic subgroups; text labels, negation, ordinal/far/arrangement relationships, and generic or coarse-grained target references are consistent weak spots across methods.
  • ViGiL3D can serve as a compact diagnostic benchmark for future 3DVG development, since human annotators solve 84% of the ScanNet prompts, showing the prompts are solvable and the models have headroom.
  • Because the dataset is small (350 prompts), its value is diagnostic rather than as a training set; the paper argues the annotation principles should be scaled for future data generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 35-metric taxonomy is a tool that could be applied to other language-to-scene benchmarks, such as embodied instruction following or 2D referring expression comprehension, to test whether datasets are accidentally narrow in the same way.
  • A natural next experiment, not run in the paper, is to generate ViGiL3D-style prompts at scale with language models conditioned on the taxonomy and train a model on them; the paper's claim predicts that such training would recover much of the lost accuracy, while adding more standard prompts would not.
  • The 350-prompt size and the simplified 25-pattern reweighting model leave open how much of the drop is due to linguistic diversity versus task artifacts like zero-target prompts and the particular scene sample; a much larger matched sample would separate these factors.
  • The near-universal failure on text labels points to a concrete architectural gap: methods that only consume point clouds cannot read writing on objects, so RGB-D fusion is a plausible route to closing one subgroup of the gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. This paper introduces ViGiL3D, a manually annotated 3D visual grounding benchmark of 350 prompts over ScanNet and ScanNet++ scenes, designed to cover linguistic patterns that prior datasets under-represent (negation, generic and coarse-grained references, non-first-noun-phrase targets, zero- and multi-target prompts, and a wider bigram diversity). It also proposes an automated pipeline combining GPT-4o scene-graph extraction with SpaCy dependency parsing to compute 35 linguistic metrics, validates that pipeline on 225 manually annotated prompts (P=0.86, R=0.91), and reports subgroup and aggregate accuracy/F1 for seven open-vocabulary 3DVG methods. The main empirical finding is that all evaluated methods perform substantially worse on ViGiL3D than on ScanRefer, which the paper interprets as evidence that current 3DVG models are not proficient at linguistically diverse, out-of-distribution prompts and that scaling data volume alone is insufficient.

Significance. If the central claim is appropriately scoped, this is a useful diagnostic resource: the prompt set is manually curated, human evaluators reach 84% accuracy on the ScanNet subset, the analysis pipeline is validated, and the consistent performance drops in Table 6 provide a worthwhile signal for the 3DVG community. The paper also makes a good-faith effort to probe alternative explanations with the reweighted evaluation in Appendix C.2, and the limitations section is unusually candid. The main weakness is that the headline conclusion attributes the performance drop specifically to linguistic diversity, while the benchmark differs from ScanRefer along several non-linguistic dimensions that are not fully controlled.

major comments (4)
  1. [Section 4, Table 5, and Table 6] The headline claim that the performance drop is due to linguistic diversity is confounded by task-format differences. Table 5 reports 43/350 prompts (12%) with zero targets and 32/350 (9%) with multiple targets, whereas the ScanRefer baselines are single-target tasks; Section 4 also states that prompts are deliberately less overspecified than typical ScanRefer descriptions. The large drops in Table 6 therefore do not isolate language. Please report results on the single-target subset (275 prompts) and/or construct a matched comparison in which target count and overspecification are held similar to ScanRefer, and frame the headline claim around the full ViGiL3D task rather than linguistic diversity alone.
  2. [Appendix C.2] The reweighting control does not support the conclusion that linguistic-pattern distribution underlies the performance drop. The paper concedes that the simplified co-occurrence model over 25 patterns 'causes a regression toward the unweighted mean' and lists zero/multi-target status and reduced overspecification as factors outside the reweighting. A control that regresses toward the unweighted mean cannot separate pattern prevalence from per-prompt difficulty. Please replace this with a more direct control, such as inverse-propensity reweighting over the full pattern set or an evaluation on a subset matched on target count and specificity, before claiming that the difficulty of ViGiL3D is attributable to linguistic diversity rather than task format.
  3. [Section 5.2 and Conclusion] The statement that 'simply scaling data volume alone is insufficient' is not directly tested. The paper compares methods with different architectures, training data, and scales (3D-VisTA vs. 3D-GRAND vs. PQ3D), but no experiment varies training-set volume while holding architecture and pattern distribution fixed. This conclusion should either be removed or supported by a controlled intervention, for example fine-tuning a single architecture on reweighted samples of increasing volume.
  4. [Section 5.1] The evaluation protocol for zero-target prompts is undefined. The paper defines accuracy for multi-target descriptions but does not state how a correct answer is scored for the 43 zero-target prompts. Without a clear scoring rule for these prompts, the aggregate Acc/F1 numbers in Tables 6 and 7 are ambiguous, and the comparison with ScanRefer is not fully well-defined. Please specify the zero-target scoring rule and, as a robustness check, report results on the single-target subset separately.
minor comments (3)
  1. [Appendix A.4] The validation-sample arithmetic is inconsistent: 20 prompts from each of the prior datasets plus 100 prompts from ViGiL3D totals more than 225 unless fewer than seven prior datasets were sampled; please make the per-dataset counts explicit.
  2. [Table 7 and Table 14] With 350 total prompts and subgroups as small as 25-30 prompts, several subgroup differences are likely within statistical noise; reporting confidence intervals or significance tests for the subgroup accuracies would strengthen the perceptual claims in Section 5.2.
  3. [Tables 4, 10, 11, and 12] The column abbreviations differ across the main table and appendices (e.g., Table 4 omits some columns shown in Table 12), making it hard to compare the full metric set in one place; please align the column names and thresholds across all tables.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ViGiL3D is an externally evaluated benchmark, not a derivation whose outputs are equivalent to its inputs.

full rationale

The paper's central empirical claim is that existing open-vocabulary 3DVG methods perform substantially worse on ViGiL3D than on ScanRefer. This claim is supported by running published, independently trained models (OpenScene, LERF, ZSVG3D, LLM-Grounder, 3D-VisTA, 3D-GRAND, PQ3D) on a manually annotated test set, with human validation at 84% accuracy (Section B.2). The dataset construction is not fitted to any baseline's predictions, and the reported accuracy drops are measured outcomes, not quantities defined by the construction. The linguistic analysis pipeline uses GPT-4o and SpaCy, but its output is validated against 225 manually annotated prompts (Section 3.1) and it is used for dataset characterization rather than for producing the model performance numbers. Some baselines use GPT-4 or GPT-4o, but this does not force their ViGiL3D accuracy values: the same model family is not the source of the benchmark labels, and the comparison to ScanRefer is external. The paper cites prior work by overlapping authors (e.g., Multi3DRefer, Zhang et al., 2023) when describing existing datasets and in noting that PQ3D is trained on an aggregate dataset including Multi3DRefer, but this citation is contextual and not load-bearing: the ViGiL3D results are not derived from Multi3DRefer's stated properties, and PQ3D's training data is a fact about the method, not an assumption smuggled in to secure the paper's conclusion. The Appendix C.2 reweighting control is admittedly limited, and the attribution of performance drops specifically to linguistic diversity is partly confounded by zero/multi-target prompts and deliberate non-overspecification; however, these are validity and confound concerns, not circularity. No equation, fitted parameter, or self-citation chain makes any 'prediction' equivalent to its input by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new theoretical entities are introduced. The free parameters are design thresholds and sample sizes. The axioms are domain assumptions about the adequacy of the linguistic taxonomy, the GPT-4o parsing tool, the scene sample, and the annotators' language, all of which are partially validated within the paper.

free parameters (2)
  • Binary metric thresholds = 5% and 20%
    Thresholds chosen by hand to classify each binary metric as absent, some, or a lot across datasets; they drive all comparative claims in Table 4 and are documented in Appendix A.3.
  • Number of prompts sampled per prior dataset = 1000
    The analysis pipeline was applied to 1000 randomly sampled prompts from each prior dataset, a sample size chosen by the authors without statistical justification for the diversity estimates.
assumptions (4)
  • domain assumption The 35 metrics and the 10 attribute and 9 relationship type taxonomies are a sufficient characterization of linguistically diverse 3DVG prompts.
    Criteria were selected based on qualitative observation of a sample of prompts (Appendix A.1); the authors themselves acknowledge that other taxonomies may exist.
  • domain assumption GPT-4o correctly parses grounding prompts into augmented scene graphs and linguistic properties.
    The pipeline is validated on 225 manually annotated prompts with average precision 0.86 and recall 0.91, so this is a tool assumption that is tested but not guaranteed for all prompt types.
  • domain assumption Scenes from the validation splits of ScanNet and ScanNet++ are representative enough to support conclusions about model generalization.
    The 35 scenes are deliberately sampled from validation splits to avoid training contamination, but indoor scene and object class coverage is limited.
  • domain assumption Prompts written by the authors are ecologically valid examples of English grounding language.
    Prompts were authored internally by a small annotator group (Appendix B.1), and human evaluators achieved 84% accuracy, so the prompts are solvable but they are not sampled from a broad population of language users.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ViGiL3D: A Linguistically Diverse Dataset for 3D Visual Grounding." pith.science (2026). https://pith.science/paper/PJSNZJJP

@misc{pith2026250101366,
  author       = {Pith},
  title        = {Pith review of: ViGiL3D: A Linguistically Diverse Dataset for 3D Visual Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PJSNZJJP}},
  note         = {Machine review of arXiv:2501.01366}
}
read the original abstract

3D visual grounding (3DVG) involves localizing entities in a 3D scene referred to by natural language text. Such models are useful for embodied AI and scene retrieval applications, which involve searching for objects or patterns using natural language descriptions. While recent works have focused on LLM-based scaling of 3DVG datasets, these datasets do not capture the full range of potential prompts which could be specified in the English language. To ensure that we are scaling up and testing against a useful and representative set of prompts, we propose a framework for linguistically analyzing 3DVG prompts and introduce Visual Grounding with Diverse Language in 3D (ViGiL3D), a diagnostic dataset for evaluating visual grounding methods against a diverse set of language patterns. We evaluate existing open-vocabulary 3DVG methods to demonstrate that these methods are not yet proficient in understanding and identifying the targets of more challenging, out-of-distribution prompts, toward real-world applications.

Figures

Figures reproduced from arXiv: 2501.01366 by the authors.

Figure 1
Figure 1. Overview of ViGiL3D. We propose a new dataset for visual grounding to better evaluate 3D visual grounding methods on the wide diversity of linguistic patterns possible to refer to objects in a scene. While existing datasets largely contain more homogenous and direct prompts, ViGiL3D includes coarse-grained ob￾ject references, negation, reference resolution, and other phenomena in more varied sentence structures whic… view at source ↗
Figure 2
Figure 2. Dataset Analysis Pipeline. Using GPT-4o and SpaCy, we automatically parse each visual grounding prompt and compute aggregate statistics for a variety of linguistic patterns for each dataset. We use GPT-4o for parsing an augmented scene graph optimized for visual grounding descriptions, and SpaCy for dependency parsing. Type Examples Color On top of the shelf is a red and yellow object. Size This is the tallest woode… view at source ↗
Figure 3
Figure 3. Examples. We show example predictions for each model on prompts with different linguistic patterns. due to the inability of CLIP to parse complex lan￾guage patterns (Yuksekgonul et al., 2023). Howev￾ercorrelation with ScanRefer performance is loose, with ZSVG3D and 3D-GRAND both outperform￾ing 3D-VisTA with GT boxes. GT vs. predicted boxes. While better perfor￾mance with ground truth information is expected, LLM-Gro… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Dataset Comparison. We visualize the pro￾portion of prompts representing different metrics up to 20% of the dataset size. In our pipeline, we use gpt-4o-2024-08-06 (Achiam et al., 2023) as our LLM to parse the grounding prompt into an augmented scene graph￾like represe…
Figure 5
Figure 5. Figure 5: 3DVG Model Architectures. Clip-aligned models align a 3D point cloud encoder with a pretrained CLIP image encoder during training, resulting in an alignment between the point encoder and CLIP text encoder. Zero-shot LLMs are used as language understanding and reasoning…
Figure 6
Figure 6. Figure 6: Model Comparison on ViGiL3D. We com￾pare the performances of models on various subgroups on the ScanNet scenes of ViGiL3D using ground truth bounding boxes. We report the subgroup analysis based on Mask3D boxes in [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Examples. We provide additional examples for prompts from ViGiL3D on ScanNet scenes. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]
Figure 8
Figure 8. Figure 8: Examples. We provide additional examples for prompts from ViGiL3D on ScanNet++ scenes. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 13 canonical work pages

  1. [1]

    Ahmed Abdelreheem, Kyle Olszewski, Hsin-Ying Lee, Peter Wonka, and Panos Achlioptas. 2024. https://arxiv.org/abs/2212.06250 ScanEnts3D : Exploiting phrase-to- 3D -object correspondences for improved visio-linguistic models in 3D scenes . In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3524--3534

    work page Pith review arXiv 2024

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. https://arxiv.org/abs/2303.08774 GPT -4 technical report . arXiv preprint arXiv:2303.08774

    arXiv 2023

  3. [3]

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. 2020. https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123460409.pdf ReferIt3D : Neural listeners for fine-grained 3D object identification in real-world scenes . In Proc. of European Conference on Computer Vision (ECCV), pages 422--440. Springer

    work page arXiv 2020

  4. [4]

    Daigang Cai, Lichen Zhao, Jing Zhang, Lu Sheng, and Dong Xu. 2022. https://openaccess.thecvf.com/content/CVPR2022/papers/Cai_3DJCG_A_Unified_Framework_for_Joint_Dense_Captioning_and_Visual_CVPR_2022_paper.pdf 3DJCG : A unified framework for joint dense captioning and visual grounding on 3D point clouds . In Proceedings of the IEEE/CVF Conference on Comput...

    work page 2022

  5. [5]

    Matthew Chang, Gunjan Chhablani, Alexander Clegg, Mikael Dallaire Cote, Ruta Desai, Michal Hlavac, Vladimir Karashchuk, Jacob Krantz, Roozbeh Mottaghi, Priyam Parashar, et al. 2024. https://arxiv.org/abs/2411.00081 Partnr: A benchmark for planning and reasoning in embodied multi-agent tasks . arXiv preprint arXiv:2411.00081

    arXiv 2024

  6. [6]

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nie ner. 2020. https://arxiv.org/abs/1912.08830 ScanRefer : 3D object localization in RGB-D scans using natural language . Proc. of European Conference on Computer Vision (ECCV)

    arXiv 2020

  7. [7]

    Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. 2022. https://arxiv.org/abs/2211.09646 Language conditioned spatial relation reasoning for 3D object grounding . Advances in neural information processing systems, 35:20522--20535

    arXiv 2022

  8. [8]

    Zhenyu Chen, Ronghang Hu, Xinlei Chen, Matthias Nie ner, and Angel X Chang. 2023. https://arxiv.org/abs/2212.00836 Unit3D : A unified transformer for 3D dense captioning and visual grounding . In Proceedings of the IEEE/CVF international conference on computer vision, pages 18109--18119

    arXiv 2023

Show all 48 references
  1. [9]

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nie ner. 2017. https://arxiv.org/abs/1702.04405 ScanNet : Richly-annotated 3D reconstructions of indoor scenes . In Proceedings of the IEEE conference on computer vision and pattern recogn...

    2017 arXiv

  2. [10]

    Alexandros Delitzas, Ayca Takmaz, Federico Tombari, Robert Sumner, Marc Pollefeys, and Francis Engelmann. 2024. https://openaccess.thecvf.com/content/CVPR2024/papers/Delitzas_SceneFun3D_Fine-Grained_Functionality_and_Affordance_Understanding_in_3D_Scenes_CVPR_2024_paper.pdf Sc...

    2024

  3. [11]

    Martin Ester, Hans-Peter Kriegel, J \"o rg Sander, Xiaowei Xu, et al. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Knowledge Discovery and Data Mining (KDD), volume 96, pages 226--231

    1996

  4. [12]

    Qiao Gu, Ali Kuwajerwala, Sacha Morin, Krishna Murthy Jatavallabhula, Bipasha Sen, Aditya Agarwal, Corban Rivera, William Paul, Kirsty Ellis, Rama Chellappa, et al. 2024. https://arxiv.org/abs/2309.16650 Conceptgraphs: Open-vocabulary 3d scene graphs for perception and plannin...

    2024 arXiv

  5. [13]

    Shuting He, Henghui Ding, Xudong Jiang, and Bihan Wen. 2024. https://arxiv.org/abs/2407.13761 SegPoint : Segment any point cloud via large language model . In Proc. of European Conference on Computer Vision (ECCV)

    2024 arXiv

  6. [14]

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 2023. https://arxiv.org/abs/2307.12981 3D-LLM : Injecting the 3D world into large language models . Advances in Neural Information Processing Systems, 36:20482--20494

    2023 arXiv

  7. [15]

    Matthew Honnibal and Ines Montani. 2017. https://spacy.io spaCy 2 : Natural language understanding with B loom embeddings, convolutional neural networks and incremental parsing

    2017

  8. [16]

    Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. 2024. An embodied generalist agent in 3D world. In International Conference on Machine Learning (ICML)

    2024

  9. [17]

    Ayush Jain, Nikolaos Gkanatsios, Ishita Mediratta, and Katerina Fragkiadaki. 2022. https://arxiv.org/abs/2112.08879 Bottom up top down detection transformers for language grounding in images and point clouds . In European Conference on Computer Vision, pages 417--433. Springer

    2022 arXiv

  10. [18]

    Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, and Siyuan Huang. 2024. https://www.ecva.net/papers/eccv_2024/papers_ECCV/papers/01407.pdf SceneVerse : Scaling 3D vision-language learning for grounded scene understanding . In Proc. of Europea...

    2024

  11. [19]

    Zhao Jin, Munawar Hayat, Yuwei Yang, Yulan Guo, and Yinjie Lei. 2023. https://openaccess.thecvf.com/content/CVPR2023/papers/Jin_Context-Aware_Alignment_and_Mutual_Masking_for_3D-Language_Pre-Training_CVPR_2023_paper.pdf Context-aware alignment and mutual masking for 3D -langua...

    2023

  12. [20]

    Shunya Kato, Shuhei Kurita, Chenhui Chu, and Sadao Kurohashi. 2023. https://aclanthology.org/2023.findings-emnlp.56/ ArkitSceneRefer : Text-based localization of small objects in diverse real-world 3D indoor scenes . In Findings of the Association for Computational Linguistics...

    2023

  13. [21]

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. 2023. https://arxiv.org/abs/2303.09553 LERF : Language embedded radiance fields . In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19729--19739

    2023 arXiv

  14. [22]

    Mingsheng Li, Xin Chen, Chi Zhang, Sijin Chen, Hongyuan Zhu, Fukun Yin, Gang Yu, and Tao Chen. 2023. https://arxiv.org/abs/2312.10763 M3DBench : Let's instruct large models with multi-modal 3D prompts . arXiv preprint arXiv:2312.10763

    2023 arXiv

  15. [23]

    Ruiyuan Lyu, Tai Wang, Jingli Lin, Shuai Yang, Xiaohan Mao, Yilun Chen, Runsen Xu, Haifeng Huang, Chenming Zhu, Dahua Lin, et al. 2024. https://arxiv.org/abs/2406.09401 MMScan : A multi-modal 3D scene dataset with hierarchical grounded language annotations . arXiv preprint arX...

    2024 arXiv

  16. [24]

    Yunze Man, Shuhong Zheng, Zhipeng Bao, Martial Hebert, Liang-Yan Gui, and Yu-Xiong Wang. 2024. https://www.arxiv.org/abs/2409.03757 Lexicon3D : Probing visual encoding models for complex 3D scene understanding . arXiv preprint arXiv:2409.03757

    2024 arXiv

  17. [25]

    Thomas Mensink, Jasper Uijlings, Lluis Castrejon, Arushi Goel, Felipe Cadar, Howard Zhou, Fei Sha, Andr \'e Araujo, and Vittorio Ferrari. 2023. https://arxiv.org/abs/2306.09224 Encyclopedic VQA : Visual questions about detailed properties of fine-grained categories . In Procee...

    2023 arXiv

  18. [26]

    Taiki Miyanishi, Fumiya Kitamori, Shuhei Kurita, Jungdae Lee, Motoaki Kawanabe, and Nakamasa Inoue. 2023. https://arxiv.org/abs/2310.18773 CityRefer : geography-aware 3D visual grounding dataset on city-scale point cloud data . arXiv preprint arXiv:2310.18773

    2023 arXiv

  19. [27]

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. 2023. https://arxiv.org/abs/2211.15654 OpenScene : 3D scene understanding with open vocabularies . In Proceedings of the IEEE/CVF conference on computer vision and pattern re...

    2023 arXiv

  20. [28]

    Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, and Yu-Gang Jiang. 2024. https://arxiv.org/abs/2305.14836 Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario . In Proceedings of the AAAI Conference on Artificial Intelligence, vol...

    2024 arXiv

  21. [29]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. https://arxiv.org/abs/2103.00020 Learning transferable visual models from natural language supervision . In Internat...

    2021 arXiv

  22. [30]

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. 2023. https://arxiv.org/abs/2210.03105 Mask3D : Mask transformer for 3D semantic instance segmentation . In IEEE International Conference on Robotics and Automation (ICRA), pages 8216-...

    2023 arXiv

  23. [31]

    Penglei Sun, Yaoxian Song, Xiang Liu, Xiaofei Yang, Qiang Wang, Tiefeng Li, Yang Yang, and Xiaowen Chu. 2024. https://arxiv.org/abs/2407.17398 3d question answering for city scene understanding . In Proceedings of the 32nd ACM International Conference on Multimedia, pages 2156--2165

    2024 arXiv

  24. [32]

    Emilia Szymanska, Mihai Dusmanu, Jan-Willem Buurlage, Mahdi Rad, and Marc Pollefeys. 2024. https://arxiv.org/abs/2408.16662 Space3D-Bench : Spatial 3D question answering benchmark . arXiv preprint arXiv:2408.16662

    2024 arXiv

  25. [33]

    Ay c a Takmaz, Elisabetta Fedele, Robert W Sumner, Marc Pollefeys, Federico Tombari, and Francis Engelmann. 2023. https://arxiv.org/abs/2306.13631 OpenMask3D : Open-vocabulary 3D instance segmentation . Advances in neural information processing systems

    2023 arXiv

  26. [34]

    Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, and Angjoo Kanazawa. 2023. https://arxiv.org/abs/2302.04264 NeRFStudio : A modular framework for neural...

    2023 arXiv

  27. [35]

    Yanmin Wu, Xinhua Cheng, Renrui Zhang, Zesen Cheng, and Jian Zhang. 2023. https://arxiv.org/abs/2209.14941 EDA : Explicit text-decoupling and dense alignment for 3D visual grounding . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1...

    2023 arXiv

  28. [36]

    Jianing Yang, Xuweiyi Chen, Nikhil Madaan, Madhavan Iyengar, Shengyi Qian, David F Fouhey, and Joyce Chai. 2024 a . https://arxiv.org/abs/2406.05132 3D-GRAND : A million-scale dataset for 3D-LLM s with better grounding and less hallucination . arXiv preprint arXiv:2406.05132

    2024 arXiv

  29. [37]

    Jianing Yang, Xuweiyi Chen, Shengyi Qian, Nikhil Madaan, Madhavan Iyengar, David F Fouhey, and Joyce Chai. 2024 b . https://arxiv.org/abs/2309.12311 LLM -grounder: Open-vocabulary 3D visual grounding with large language model as an agent . In 2024 IEEE International Conference...

    2024 arXiv

  30. [38]

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nie ner, and Angela Dai. 2023. https://arxiv.org/abs/2308.11417 ScanNet++ : A high-fidelity dataset of 3D indoor scenes . In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12--22

    2023 arXiv

  31. [39]

    Zhihao Yuan, Jinke Ren, Chun-Mei Feng, Hengshuang Zhao, Shuguang Cui, and Zhen Li. 2024. https://arxiv.org/abs/2311.15383 Visual programming for zero-shot open-vocabulary 3D visual grounding . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    2024 arXiv

  32. [40]

    Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. 2023. https://arxiv.org/abs/2210.01936 When and why vision-language models behave like bags-of-words, and what to do about it? In Proc. of International Conference on Learning Representations (ICLR)

    2023 arXiv

  33. [41]

    Haochen Zhang, Nader Zantout, Pujith Kachana, Zongyuan Wu, Ji Zhang, and Wenshan Wang. 2024 a . https://arxiv.org/abs/2411.03540 VLA-3D : A dataset for 3D semantic scene understanding and navigation . arXiv preprint arXiv:2403.09631

    2024 arXiv

  34. [42]

    Yiming Zhang, ZeMing Gong, and Angel X Chang. 2023. https://arxiv.org/abs/2309.05251 Multi3Drefer : Grounding text description to multiple 3D objects . In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15225--15236

    2023 arXiv

  35. [43]

    Zhuofan Zhang, Ziyu Zhu, Pengxiang Li, Tengyu Liu, Xiaojian Ma, Yixin Chen, Baoxiong Jia, Siyuan Huang, and Qing Li. 2024 b . https://arxiv.org/abs/2408.04034 Task-oriented sequential grounding in 3D scenes . arXiv preprint arXiv:2408.04034

    2024 arXiv

  36. [44]

    Chenming Zhu, Tai Wang, Wenwei Zhang, Kai Chen, and Xihui Liu. 2024 a . https://arxiv.org/abs/2407.01525 ScanReason : Empowering 3D visual grounding with reasoning capabilities . In Proc. of European Conference on Computer Vision (ECCV)

    2024 arXiv

  37. [45]

    Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, and Qing Li. 2023. https://arxiv.org/abs/2308.04352 3D-VisTA : Pre-trained transformer for 3D vision and text alignment . In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2911--2921

    2023 arXiv

  38. [46]

    Ziyu Zhu, Zhuofan Zhang, Xiaojian Ma, Xuesong Niu, Yixin Chen, Baoxiong Jia, Zhidong Deng, Siyuan Huang, and Qing Li. 2024 b . https://arxiv.org/abs/2405.11442 Unifying 3D vision-language understanding via promptable queries . In Proc. of European Conference on Computer Vision (ECCV)

    2024 arXiv

  39. [47]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.