Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problems

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that reformulating zero-shot 3D visual grounding as a constraint satisfaction problem, with all spatial relations solved together, outperforms prior LLM-based step-by-step methods by a wide margin on two public benchmarks.

desk verdict A clean CSP reformulation for zero-shot 3D grounding with a plausible empirical story, but the headline margins are confounded by an LLM switch and eval-set tuning. read the letter →

arxiv 2411.14594 v2 pith:H4YHRRKD submitted 2024-11-21 cs.CV

Qihao Yuan , Kailai Li , Jiaming Zhang This is my paper · ORCID
classification cs.CV
keywords 3Dvisualgroundingzero-shotconstraintsatisfactionproblemlargelanguagemodelsspatialreasoningpointcloudnegationandcountingqueries
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a zero-shot 3D visual grounding system works better when it treats a natural-language query as a constraint satisfaction problem: every mentioned object becomes a variable and every spatial relation becomes a constraint, and a solver finds an assignment that satisfies all of them at once. Prior LLM-based methods reason about one pairwise relation at a time, which can lock in a wrong choice before later relations are considered. The authors report that their system, CSVG, raises Acc@0.5 accuracy over the previous state of the art by 11.2 points on Nr3D and 7.0 on ScanRefer, using only an open-source LLM. If correct, this shows that global symbolic reasoning about the whole scene is a more reliable route than local filtering for open-vocabulary grounding.

What carries the argument

The central object is the CSP built from the LLM-generated program: variables are object mentions with domains restricted to instances of matching labels, and constraints are drawn from a predefined library of spatial-relation functions (e.g., CONSTRAINT_BESIDE, CONSTRAINT_BETWEEN) plus min/max constraints and negative variables. The solver is backtracking with the min/max constraints applied after spatial constraints, followed by a heuristic that picks the solution with the minimum average pairwise distance between its objects.

What would settle it

Measure the program-generation success rate directly on a random sample of queries: extract the CSP the LLM produces, solve it, and compare to human-annotated groundings. If a large fraction of generated programs do not express the query's stated relations, or if solver output does not track human judgments when programs are correct, the claimed mechanism would be refuted.

Watch

Extended reading notes

Core claim

Reformulating 3DVG as a CSP means the LLM generates a small Python program that declares variables for the objects mentioned in the query, applies predefined constraint functions for spatial relations such as 'beside', 'on', or 'between', and optionally marks a variable as negative for negation queries or adds min/max constraints for superlatives. A backtracking solver then searches over all instances with matching labels, and the final assignment simultaneously satisfies every constraint, yielding both the target and the anchor objects. The authors claim this global satisfaction step is the main source of their accuracy gain: removing it and reverting to single-relation filtering drops Acc@0.5 below the previous state of the art, while adding it surpasses that baseline. Counting queries ('the third chair') are handled by creating one variable per counted object and ordering them with comparison constraints; negation queries are handled by testing each candidate assignment against negative variables and rejecting solutions where a forbidden relation holds.

Load-bearing premise

The entire pipeline depends on the language model reliably translating natural-language spatial relations into correct calls to the predefined constraint functions; if it generates the wrong relations or misses one, the solver faithfully returns a wrong answer.

Editorial extensions

If this is right

  • If the claim holds, a zero-shot system can match or exceed several supervised approaches on ScanRefer and Nr3D without any training data for grounding.
  • Because the solver returns both target and anchor objects, downstream tasks that need context objects, such as robot grasp planning or navigation, get additional information for free.
  • Negation and counting queries, which supervised methods handle only with extra training data, can be added with a few lines of solver code and a prompt example.
  • The gains come despite using a smaller open-source LLM (Mistral-Large-2407) than the GPT-4 used by prior baselines, suggesting headroom with stronger models.
  • Accuracy jumps from 39.8 to 61.6 Acc@0.5 on ScanRefer when ground-truth segmentation replaces Mask3D predictions, indicating that segmentation quality, not just reasoning, bounds performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the constraint library uses fixed distance thresholds, adapting to a new dataset or sensor scale probably requires re-tuning those thresholds, as the appendix itself notes; a version that learns or self-adapts thresholds per scene would be a natural next step.
  • The minimum-average-distance heuristic assumes referenced objects are usually close together; queries about isolated or deliberately far objects (e.g., 'the one far from everything') would likely break that heuristic and deserve a targeted experiment.
  • The same CSP formulation should transfer to 2D referring expression comprehension or 3D caption-guided navigation, where the constraint library would operate on bounding boxes or occupancy maps instead of point-cloud instances.
  • Letting the LLM invent constraint functions on the fly, flagged as future work, could remove the bottleneck of a fixed vocabulary of relations but would also remove the validation safety that the predefined library provides.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CSVG, a zero-shot 3D visual grounding method that reformulates the task as a Constraint Satisfaction Problem. An LLM generates a Python program that defines variables for relevant objects and constraints for their spatial relations; a backtracking solver then finds assignments satisfying all constraints globally, simultaneously grounding target and anchor objects. The framework is extended to negation- and counting-based queries with small solver modifications. The method is evaluated on ScanRefer and Nr3D with Mask3D and ground-truth segmentations, reporting Acc@0.5 improvements of +7.0 and +11.2 over ZSVG3D, and the authors make code and full prompts available.

Significance. If the reported gains are attributable to the proposed formulation, this is a useful contribution: global symbolic reasoning over all spatial relations is a principled alternative to the local, one-relation-at-a-time reasoning of ZSVG3D, and the CSP framing naturally supports negation and counting. The paper is also commendably concrete: it provides the full LLM prompt with eleven in-context examples, the solver algorithms, and publicly available code, which makes the system reproducible. However, the empirical evidence as presented does not yet isolate the contribution of the CSP formulation, because the comparison is confounded by a different LLM, a substantially different prompt, and evaluation-set tuning of thresholds and heuristics. The central algorithmic idea is defensible, but the evaluation needs strengthening before the stated claim can be accepted.

major comments (4)
  1. [Tables 1 and 2, Section 4.1] The headline comparisons are confounded by the LLM choice: CSVG uses Mistral-Large-2407 while ZSVG3D and LLM-Grounder use GPT-4 and VLM-Grounder uses GPT-4V. Because the system's output is LLM-generated Python programs, the +11.2 and +7.0 margins combine the proposed CSP formulation with a different LLM and an eleven-example prompt (Appendix A). To support the claim that global CSP reasoning is responsible for the improvement, the authors should provide a same-LLM comparison, e.g., running ZSVG3D with Mistral-Large-2407 or CSVG with GPT-4, or otherwise quantifying how much of the gap is due to the LLM and prompt.
  2. [Appendix B.1, Table 3(b)] The geometric thresholds (NEAR_DISTANCE, FAR_DISTANCE, ABOVE_BELOW_HORIZONTAL_DISTANCE, BETWEEN_DISTANCE) are stated to be adjusted per dataset and set to large values for ScanRefer and Nr3D because this produced the best results, and the Minimum Average Distance solution-selection heuristic is selected using the full ScanRefer validation set. This is test-set tuning of both the spatial predicates and the tie-breaking rule, which optimistically biases the reported accuracies and the margins over baselines. The paper should either tune on a separate split, report cross-validated selection, or provide a sensitivity analysis showing that the conclusions are robust to these choices.
  3. [Table 3(a)] The ablation of the global-satisfaction component is run on only 1000 ScanRefer samples, without variance or significance reporting. The +4.3 Acc@0.5 gain from global satisfaction on this subset is smaller than the +7.0 full-set margin, while Table 3(b) shows the solution-selection heuristic alone moves Acc@0.5 by +4.2 on the full set. The evidence that global constraint satisfaction, rather than the heuristic or LLM strength, carries the reported improvement is therefore not as strong as the abstract suggests; a full-set ablation or a clear statistical justification for the subset is needed.
  4. [Section 3.2, Listing 2] The system depends on the LLM generating executable and semantically correct programs that map natural-language spatial relations to the predefined constraint functions, but no analysis of program-generation success is provided. Systematic LLM errors in variable definition, label selection, or constraint choice would directly lower grounding accuracy regardless of the CSP solver. Reporting the execution success rate, the frequency of constraint-type misassignment, and representative failure modes is necessary to separate the LLM component from the CSP reasoning component.
minor comments (5)
  1. [Figure 1] The caption says that CSVG 'successfully locates the target (trash can)' for the second query, but the query is 'the nightstand without any trash can beside it'; the target should be the nightstand, with the trash can as a negative anchor. The caption should be corrected.
  2. [Listing 13] In in-context example 11, CONSTRAINT_UNDER(target=TRASH_CAN_1, anchor=COUNTER_0) references TRASH_CAN_1, but only TRASH_CAN_0 is defined in that example; this appears to be a bug in the prompt and should be fixed.
  3. [Table 2] The table header contains the typos 'repsec- tively' and 'embolded'; please revise to 'respectively' and 'bolded'.
  4. [Various] There are several typographical errors, including 'le f t' in Section 3.3, 'theses capabilities' in Section 1, 'faciliates' in Appendix B.1, and 'Satisfication' in the system prompt of Listing 2; a copyedit pass is recommended.
  5. [References] References [32] and [33] are the same 3D-VisTA paper and should be merged into a single entry.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CSVG is an empirical system evaluated against external benchmarks; no prediction reduces to a fitted parameter or self-citation by construction.

full rationale

The paper's central claim is an empirical accuracy comparison, not a derivation. The CSP formulation (variables from LLM generation, predefined spatial-relation constraints, backtracking solver, min/max constraints, and solution-selection heuristic) is defined independently of the Nr3D/ScanRefer metrics, and the reported Acc@0.25/Acc@0.5 numbers are measured on public validation sets against external baselines. The main comparisons (Tables 1 and 2) involve a different LLM than ZSVG3D (Mistral-Large-2407 vs GPT-4), which is a fairness/attribution concern about the +11.2/+7.0 margins, but it is not circular reasoning: the results could have been worse and are not forced by construction. The ablation in Table 3a directly tests global vs local constraint satisfaction on 1000 ScanRefer samples. Appendix B.1 states that threshold values 'are adjusted in accordance to the given datasets or scenarios' and Table 3b selects the solution-selection heuristic on the full ScanRefer validation set; this is test-set tuning and raises overfitting/correctness risk, but it does not make the benchmark numbers equal to the tuning criterion by definition. The paper also openly notes the subset-bias limitation in Section 4.2. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The method is self-contained and falsifiable against external benchmarks, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method assumes LLM program generation fidelity, reliable instance segmentation, and that predefined spatial constraints with dataset-specific thresholds are sufficient. No free parameter is a fitted numerical constant in the physics sense, but the thresholds and heuristic are tuned on the evaluation data, which is a form of fitting.

free parameters (2)
  • Spatial relation thresholds (NEAR_DISTANCE, FAR_DISTANCE, ABOVE_BELOW_HORIZONTAL_DISTANCE, BETWEEN_DISTANCE) = not given numerically; set to 'large values'
    Used in all spatial constraints; the paper states they are adjusted per dataset and large values worked best, implying tuning on the validation set.
  • Solution selection heuristic: Minimum Average Distance = minimum average pairwise distance among objects in a solution
    Selected as the default after comparing random, maximum, and minimum average distance on the full ScanRefer validation set; this is a post-hoc choice on the evaluation set.
assumptions (4)
  • domain assumption The LLM reliably translates natural-language queries into valid Python programs using only the predefined constraint functions.
    The entire pipeline depends on the LLM's program generation; no success-rate or failure analysis is reported. Invoked in Section 3.2 and the prompt in Appendix A.
  • domain assumption Instance segmentation provides a complete and correctly labeled list of objects.
    CSVG operates on the segmentation output; missed objects cause incorrect grounding, as acknowledged in the Fig. 4 failure case (Mask3D missed the upper table).
  • domain assumption The predefined spatial constraints and fixed thresholds are sufficient to represent the spatial semantics of the queries.
    The paper approximates relations (e.g., FRONT and BEHIND are handled as NEAR; compass directions mapped to NEAR) and tunes thresholds per dataset, a simplification that may not transfer.
  • domain assumption The CSP solver's backtracking search is computationally feasible on the generated CSPs.
    No complexity analysis is given; the solver enumerates combinations of variable assignments, which could explode with many instances, though typical scenes are small.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problems." pith.science (2026). https://pith.science/paper/H4YHRRKD

@misc{pith2026241114594,
  author       = {Pith},
  title        = {Pith review of: Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H4YHRRKD}},
  note         = {Machine review of arXiv:2411.14594}
}
abstract

3D visual grounding (3DVG) aims to locate objects in a 3D scene with natural language descriptions. Supervised methods have achieved decent accuracy, but have a closed vocabulary and limited language understanding ability. Zero-shot methods utilize large language models (LLMs) to handle natural language descriptions, where the LLM either produces grounding results directly or generates programs that compute results (symbolically). In this work, we propose a zero-shot method that reformulates the 3DVG task as a Constraint Satisfaction Problem (CSP), where the variables and constraints represent objects and their spatial relations, respectively. This allows a global symbolic reasoning of all relevant objects, producing grounding results of both the target and anchor objects. Moreover, we demonstrate the flexibility of our framework by handling negation- and counting-based queries with only minor extra coding efforts. Our system, Constraint Satisfaction Visual Grounding (CSVG), has been extensively evaluated on the public datasets ScanRefer and Nr3D datasets using only open-source LLMs. Results show the effectiveness of CSVG and superior grounding accuracy over current state-of-the-art zero-shot 3DVG methods with improvements of $+7.0\%$ (Acc@0.5 score) and $+11.2\%$ on the ScanRefer and Nr3D datasets, respectively. The code of our system is available at https://asig-x.github.io/csvg_web.

Figures

Figures reproduced from arXiv: 2411.14594 by the authors.

Figure 1
Figure 1. Comparison of conventional 3DVG and our proposed Constraint Satisfaction Visual Grounding (CSVG) framework. On [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of CSVG framework. Given a point cloud, we first perform instance segmentation to obtain a list [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. (a) Illustration of combining min/max and spatial constraints. Suppose we have one cup (icup_0) on the shelf (ishel f _0) and two cups (icup_1 and icup_2) on the table (itable_0). A CSP is formulated given the query “the largest cup on the table”. The size of the orange boxes enclosing the cups indicates their respective size, with icup_0 being the largest. Selecting the largest cup (icup_0) first leads to failure, … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Qualitative evaluation results given five queries. We denote ground truth with [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Demonstrations of our proposed CSVG in comparison with ZSVG3Di.e.. Ground truth target: [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Demonstration of our proposed CSVG in comparison with ZSVG3Di.e.. Ground truth target: [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Demonstration of the proposed CSVG with only spatial constraints. Distractors (objects with the same label as the target): [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Demonstration of the proposed CSVG with min/max constraints. Distractors (object with the same label as the target): [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Demonstration of CSVG with negative variables. Distractors (object with the same label as the target): [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Demonstration of the proposed CSVG with counting-based queries. Distractors (object with the same label as the target): [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Grounding

    cs.CV 2025-06 conditional novelty 6.0 of 10

    SPAZER, a VLM-driven agent, combines 3D rendered views with 2D camera images in a progressive pipeline to achieve state-of-the-art zero-shot 3D visual grounding.

  2. Language-to-Space Programming for Training-Free 3D Visual Grounding

    cs.CV 2025-02 conditional novelty 6.0 of 10

    LaSP uses LLM-generated Python relation encoders, tuned against small test suites, to do cheap and accurate training-free 3D visual grounding.

Reference graph

Works this paper leans on

45 extracted references · 25 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 422–440. Springer, 2020

    2020

  3. [3]

    Scanrefer: 3d object localization in rgb-d scans using natural language

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language. In European conference on computer vision, pages 202–221. Springer, 2020

    2020

  4. [4]

    Can 3d vision-language models truly understand natural language? arXiv preprint arXiv:2403.14760, 2024

    Weipeng Deng, Runyu Ding, Jihan Yang, Jiahui Liu, Yijiang Li, Xiaojuan Qi, and Edith Ngai. Can 3d vision-language models truly understand natural language? arXiv preprint arXiv:2403.14760, 2024

    arXiv 2024

  5. [5]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  6. [6]

    Visual programming: Compositional visual reasoning without training

    Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14953–14962, 2023

    work page 2023

  7. [7]

    Text-guided graph neural networks for referring 3d instance segmentation

    Pin-Hao Huang, Han-Hung Lee, Hwann-Tzong Chen, and Tyng-Luh Liu. Text-guided graph neural networks for referring 3d instance segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1610–1618, 2021

    2021

  8. [8]

    Bottom up top down detection transformers for language grounding in images and point clouds

    Ayush Jain, Nikolaos Gkanatsios, Ishita Mediratta, and Katerina Fragkiadaki. Bottom up top down detection transformers for language grounding in images and point clouds. In European Conference on Computer Vision, pages 417–433. Springer, 2022

    2022

Show all 45 references
  1. [9]

    Lerf: Language embedded radiance fields

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19729–19739, 2023

    2023

  2. [10]

    A survey on text-guided 3d visual grounding: Elements, recent advances, and future directions

    Daizong Liu, Yang Liu, Wencan Huang, and Wei Hu. A survey on text-guided 3d visual grounding: Elements, recent advances, and future directions. arXiv preprint arXiv:2406.05785, 2024

    2024 arXiv

  3. [11]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  4. [12]

    3d-sps: Single-stage 3d visual grounding via referred point progressive selection

    Junyu Luo, Jiahui Fu, Xianghao Kong, Chen Gao, Haibing Ren, Hao Shen, Huaxia Xia, and Si Liu. 3d-sps: Single-stage 3d visual grounding via referred point progressive selection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16454–16...

    2022

  5. [13]

    OpenAI. GPT-4V. Technical report, OpenAI, 2023. URL https://openai.com/index/gpt-4v-system-card/ . Technical Report

    2023

  6. [14]

    OpenAI. GPT-4o. https://openai.com/index/hello-gpt-4o/ , 2024

    2024

  7. [15]

    Openscene: 3d scene understanding with open vocabularies

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 815–824, 2023

    2023

  8. [16]

    Artificial intelligence: a modern approach

    Stuart J Russell and Peter Norvig. Artificial intelligence: a modern approach. Pearson, 2016

    2016

  9. [17]

    Mask3D: Mask Transformer for 3D Semantic Instance Segmentation

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. Mask3D: Mask Transformer for 3D Semantic Instance Segmentation. 2023

    2023

  10. [18]

    Vipergpt: Visual inference via python execution for reasoning

    Dídac Surís, Sachit Menon, and Carl V ondrick. Vipergpt: Visual inference via python execution for reasoning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11888–11898, 2023

    2023

  11. [19]

    Mistral large 2: The new generation of flagship model

    Mistral AI team. Mistral large 2: The new generation of flagship model. https://mistral.ai/news/ mistral-large-2407/, July 2024. Accessed: 2024-10-14

    2024

  12. [20]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polo- sukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  13. [21]

    Gˆ 3-lq: Marrying hyperbolic alignment with explicit semantic-geometric modeling for 3d visual grounding

    Yuan Wang, Yali Li, and Shengjin Wang. Gˆ 3-lq: Marrying hyperbolic alignment with explicit semantic-geometric modeling for 3d visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13917–13926, 2024

    2024

  14. [22]

    Dynamic graph cnn for learning on point clouds

    Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog), 38(5):1–12, 2019

    2019

  15. [23]

    3drp-net: 3d relative position-aware network for 3d visual grounding

    Zehan Wang, Haifeng Huang, Yang Zhao, Linjun Li, Xize Cheng, Yichen Zhu, Aoxiong Yin, and Zhou Zhao. 3drp-net: 3d relative position-aware network for 3d visual grounding. arXiv preprint arXiv:2307.13363, 2023

    2023 arXiv

  16. [24]

    Eda: Explicit text-decoupling and dense alignment for 3d visual grounding

    Yanmin Wu, Xinhua Cheng, Renrui Zhang, Zesen Cheng, and Jian Zhang. Eda: Explicit text-decoupling and dense alignment for 3d visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19231–19242, 2023. 8 QIHAO YUAN, KAILAI L...

    2023

  17. [25]

    VLM-grounder: A VLM agent for zero- shot 3d visual grounding

    Runsen Xu, Zhiwei Huang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. VLM-grounder: A VLM agent for zero- shot 3d visual grounding. In 8th Annual Conference on Robot Learning, 2024. URLhttps://openreview.net/forum? id=IcOrwlXzMi

    2024

  18. [26]

    Llm- grounder: Open-vocabulary 3d visual grounding with large language model as an agent

    Jianing Yang, Xuweiyi Chen, Shengyi Qian, Nikhil Madaan, Madhavan Iyengar, David F Fouhey, and Joyce Chai. Llm- grounder: Open-vocabulary 3d visual grounding with large language model as an agent. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 76...

    2024

  19. [27]

    Visual programming for zero- shot open-vocabulary 3d visual grounding

    Zhihao Yuan, Jinke Ren, Chun-Mei Feng, Hengshuang Zhao, Shuguang Cui, and Zhen Li. Visual programming for zero- shot open-vocabulary 3d visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20623–20633, 2024

    2024

  20. [28]

    Multi3drefer: Grounding text description to multiple 3d objects

    Yiming Zhang, ZeMing Gong, and Angel X Chang. Multi3drefer: Grounding text description to multiple 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15225–15236, 2023

    2023

  21. [29]

    3dvg-transformer: Relation modeling for visual grounding on point clouds

    Lichen Zhao, Daigang Cai, Lu Sheng, and Dong Xu. 3dvg-transformer: Relation modeling for visual grounding on point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2928–2937, 2021

    2021

  22. [30]

    Efficiently programming large language models using sglang

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Jeff Huang, Chuyue Sun, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Efficiently programming large language models using sglang. arXiv preprint arXiv:2312.07104, 2023

    2023 arXiv

  23. [31]

    Graph neural networks: A review of methods and applications

    Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI open, 1:57–81, 2020

    2020

  24. [32]

    3d-vista: Pre-trained transformer for 3d vision and text alignment

    Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, and Qing Li. 3d-vista: Pre-trained transformer for 3d vision and text alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2911–2921, 2023

    2023

  25. [33]

    system”, “user

    Ziyu Zhu, Xiaojian Ma, Yixin Chen, Zhidong Deng, Siyuan Huang, and Qing Li. 3d-vista: Pre-trained transformer for 3d vision and text alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2911–2921, 2023. QIHAO YUAN, KAILAI LI, JIAMING ZHA...

    2023

  26. [35]

    chair"]) WINDOW_0 = DEFINE_VARIABLE(labels=[

    coffee table <[ASSISTANT]> # define variables for the CSP CHAIR_0 = DEFINE_VARIABLE(labels=["chair"]) WINDOW_0 = DEFINE_VARIABLE(labels=["window"]) COFFEE_TABLE_0 = DEFINE_VARIABLE(labels=["coffee table"]) # add a constraint: the grey chair is at the end of the shorter coffee ...

  27. [36]

    desk"]) DESK_LEFT = DEFINE_VARIABLE(labels=[

    desk <[ASSISTANT]> # since there are three desks in total, I’ll create an object to represent each of them DESK_MIDDLE = DEFINE_VARIABLE(labels=["desk"]) DESK_LEFT = DEFINE_VARIABLE(labels=["desk"]) DESK_RIGHT = DEFINE_VARIABLE(labels=["desk"]) # the target desk has one desk t...

  28. [37]

    chair"]) # define the room center variable ROOM_CENTER = DEFINE_VARIABLE(labels=[

    chair <[ASSISTANT]> # define a variable for the target chair object CHAIR_0 = DEFINE_VARIABLE(labels=["chair"]) # define the room center variable ROOM_CENTER = DEFINE_VARIABLE(labels=["room center"]) # add a constraint: the chair is near the room center # we pass the set of al...

  29. [38]

    trash can

    trash can <[ASSISTANT]> # define a variable for the blue trash can (target) TRASH_CAN_BLUE = DEFINE_VARIABLE(labels=["trash can"]) # define a variable for the orange trash can TRASH_CAN_ORANGE = DEFINE_VARIABLE(labels=["trash can"]) # define a variable for the far away trash c...

  30. [39]

    Global Satis

    to determine the final solution. Given this solution, a bounding box is produced for grounding the target (line 40). Grounding through Local Constraints. Besides the CSP solver described previously, we have also implemented a separate solver that only considers a single constr...

  31. [40]

    table"]) # define a variable for the smallest cup on the largest table CUP_0 = DEFINE_VARIABLE(labels=[

    table <[ASSISTANT]> # define a variable for the largest table TABLE_0 = DEFINE_VARIABLE(labels=["table"]) # define a variable for the smallest cup on the largest table CUP_0 = DEFINE_VARIABLE(labels=["cup"]) # constraint: the largest table CONSTRAINT_MAX_OF(target=TABLE_0, sco...

  32. [41]

    chair"]) CHAIR_1 = DEFINE_VARIABLE(labels=[

    chair <[ASSISTANT]> # this is an example of how to handle counting-based query # since the target is the third chair from the left, there must be at least three chairs in total # so we define a variable for each of them CHAIR_0 = DEFINE_VARIABLE(labels=["chair"]) CHAIR_1 = DEF...

  33. [42]

    chair"]) CHAIR_1 = DEFINE_VARIABLE(labels=[

    room corner <[ASSISTANT]> # this is another example of how to handle counting-based query # it is also a comparison-based query # since the target is the third chair from the room corner, there must be at least three chairs in total # so we define a variable for each of them C...

  34. [43]

    wall"]) WINDOW_0 = DEFINE_VARIABLE(labels=[

    window <[ASSISTANT]> # define variables for the window and the wall WALL_0 = DEFINE_VARIABLE(labels=["wall"]) WINDOW_0 = DEFINE_VARIABLE(labels=["window"]) # the windows should be near the wall CONSTRAINT_NEAR(target=WINDOW_0, anchor=WALL_0) # the target window should be in th...

  35. [44]

    chair"]) # define a negative variable to express that the chair has no teddy bear on it TEDDY_BEAR_NEG = DEFINE_NEGATIVE_VARIABLE(labels=[

    teddy bear <[ASSISTANT]> # this is an example of using negative variables # define variables for the chair and the teddy bear CHAIR_0 = DEFINE_VARIABLE(labels=["chair"]) # define a negative variable to express that the chair has no teddy bear on it TEDDY_BEAR_NEG = DEFINE_NEGA...

  36. [45]

    cup"]) # define the table variable as negative, since the cup variable is the target and we want to locate it TABLE_NEG_0 = DEFINE_NEGATIVE_VARIABLE(labels=[

    table <[ASSISTANT]> # this is another example of using negative variables # define the cup variable, which is the target CUP_0 = DEFINE_VARIABLE(labels=["cup"]) # define the table variable as negative, since the cup variable is the target and we want to locate it TABLE_NEG_0 =...

  37. [46]

    trash can

    wastebin <[ASSISTANT]> # define variables for the trash can. since we have multiple labels denoting trash cans (with synonyms), we include all of them in the label. TRASH_CAN_0 = DEFINE_VARIABLE(labels=["trash can", "plastic trash bin", "wastebin"]) # define variables for the ...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.