pith. machine review for the scientific record. sign in

arxiv: 2604.26261 · v1 · submitted 2026-04-29 · 💻 cs.CV

Recognition: unknown

Multiple Consistent 2D-3D Mappings for Robust Zero-Shot 3D Visual Grounding

Yufei Yin , Jie Zheng , Qianke Meng , Zhou Yu , Minghao Chen , Jiajun Ding , Min Tan , Yuling Xi
show 2 more authors
Zhiwen Chen Chengfei Lv
Authors on Pith no claims yet

Pith reviewed 2026-05-07 13:43 UTC · model grok-4.3

classification 💻 cs.CV
keywords zero-shot 3D visual grounding2D-3D consistencyScanRefervision-language modelssemantic alignmentinstance rectificationviewpoint distillation
0
0 comments X

The pith

MCM-VG establishes multiple consistent 2D-3D mappings to achieve robust zero-shot 3D visual grounding.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to demonstrate that enforcing explicit consistency between 2D images and 3D scenes across semantics, instances, and viewpoints overcomes the core bottlenecks of inaccurate categories and imprecise geometries in open-vocabulary 3D proposals. This matters for open-world embodied AI, where systems must locate objects described in natural language inside 3D environments without any task-specific training data. The approach corrects mismatches through LLM-guided query parsing and 2D-3D matching, repairs missing 3D geometry by back-projecting VLM segmentations, and removes redundant multi-view reasoning by clustering camera directions into optimal frames paired with bird's-eye-view maps for VLM multiple-choice decisions.

Core claim

MCM-VG achieves robust zero-shot 3D visual grounding by explicitly establishing multiple consistent 2D-3D mappings. A Semantic Alignment module corrects category mismatches via LLM-driven query parsing and coarse-to-fine matching. An Instance Rectification module uses VLM-guided 2D segmentations to reconstruct missing targets and back-project accurate 3D geometries. A Viewpoint Distillation module clusters camera directions to select optimal frames, which are then paired with bird's-eye-view maps into concise visual prompts that turn target disambiguation into a multiple-choice task for vision-language models.

What carries the argument

Multiple Consistent 2D-3D Mappings, enforced by three modules that align semantics, rectify instances, and distill viewpoints to link 2D visual priors directly to 3D geometry and reasoning.

Load-bearing premise

VLM-guided 2D segmentations and LLM-driven parsing will reliably correct category mismatches and reconstruct accurate 3D geometries without introducing new errors that propagate through the mappings.

What would settle it

A controlled test showing that performance gains vanish or reverse when VLM 2D segmentations add errors on scenes containing ambiguous categories or incomplete 3D proposals.

Figures

Figures reproduced from arXiv: 2604.26261 by Chengfei Lv, Jiajun Ding, Jie Zheng, Minghao Chen, Min Tan, Qianke Meng, Yufei Yin, Yuling Xi, Zhiwen Chen, Zhou Yu.

Figure 1
Figure 1. Figure 1: Comparisons with zero-shot 3DVG methods. Com view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the MCM-VG pipeline for zero-shot 3D visual grounding. MCM-VG contains three modules for compre view at source ↗
Figure 3
Figure 3. Figure 3: Visualization results of 3D visual grounding on the ScanRefer[5] dataset. view at source ↗
Figure 4
Figure 4. Figure 4: Ablation studies of different hyperparameters. view at source ↗
Figure 5
Figure 5. Figure 5: Failure cases of our framework under severe occlu view at source ↗
Figure 6
Figure 6. Figure 6: More visualization results of 3D visual grounding on the ScanRefer [5] dataset. view at source ↗
read the original abstract

Zero-shot 3D Visual Grounding (3DVG) is a critical capability for open-world embodied AI. However, existing methods are fundamentally bottlenecked by the poor quality of open-vocabulary 3D proposals, suffering from inaccurate categories and imprecise geometries, as well as the spatial redundancy of exhaustive multi-view reasoning. To address these challenges, we propose MCM-VG, a novel framework that achieves robust zero-shot 3DVG by explicitly establishing Multiple Consistent 2D-3D Mappings. Instead of passively relying on noisy 3D segments, MCM-VG enforces 2D-3D consistency across three fundamental dimensions to achieve precise target localization and reliable reasoning. First, a Semantic Alignment module corrects category mismatches via LLM-driven query parsing and coarse-to-fine 2D-3D matching. Second, an Instance Rectification module leverages VLM-guided 2D segmentations to reconstruct missing targets, back-projecting these reliable visual priors to establish accurate 3D geometries. Finally, to eliminate spatial redundancy, a Viewpoint Distillation module clusters 3D camera directions to extract optimal frames. By pairing these optimal RGB frames with Bird's Eye View maps into concise visual prompt sets, we formulate the final target disambiguation as a multiple-choice reasoning task for Vision-Language Models. Extensive evaluations on ScanRefer and Nr3D benchmarks demonstrate that MCM-VG sets a new state-of-the-art for zero-shot 3D visual grounding. Remarkably, it achieves 62.0\% and 53.6\% in Acc@0.25 and Acc@0.5 on ScanRefer, outperforming previous baselines by substantial margins of 6.4\% and 4.0\%.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes MCM-VG, a framework for zero-shot 3D visual grounding that enforces multiple consistent 2D-3D mappings via three modules: Semantic Alignment (LLM-driven query parsing and coarse-to-fine 2D-3D matching to correct category mismatches), Instance Rectification (VLM-guided 2D segmentations back-projected to reconstruct accurate 3D geometries), and Viewpoint Distillation (clustering camera directions to select optimal frames paired with BEV maps for VLM multiple-choice reasoning). It reports new state-of-the-art results on ScanRefer (62.0% Acc@0.25 and 53.6% Acc@0.5, outperforming baselines by 6.4% and 4.0%) and Nr3D.

Significance. If the reported gains are shown to arise from the consistency mechanisms rather than implementation choices, the work would advance zero-shot 3DVG by directly addressing noisy open-vocabulary 3D proposals through 2D priors, with potential benefits for embodied AI. The explicit multi-dimensional consistency approach and substantial benchmark margins are strengths.

major comments (3)
  1. [§3.2] §3.2 (Instance Rectification): The back-projection of VLM 2D masks to 3D point clouds is presented without any quantitative breakdown of correction success rate versus introduced error rate (e.g., from depth misalignment, occlusion, or VLM hallucination), which is load-bearing for the central claim that this module yields net-positive geometry corrections.
  2. [§4] §4 (Experiments): No ablation studies are reported that isolate the contribution of Semantic Alignment, Instance Rectification, and Viewpoint Distillation individually, leaving open whether the 6.4% and 4.0% gains on ScanRefer derive from the proposed consistency mappings or from unstated tuning, baseline re-implementations, or other factors.
  3. [§3.1] §3.1 (Semantic Alignment): The LLM-driven query parsing and coarse-to-fine matching lack analysis of category misalignment propagation through the pipeline, which directly affects the reliability of the final grounding when LLM parsing errors occur.
minor comments (2)
  1. [Abstract] The abstract and §4 could explicitly name the prior zero-shot baselines used for the reported margins to facilitate direct comparison.
  2. [§3.3] Notation for the visual prompt sets in Viewpoint Distillation could be clarified with a diagram or pseudocode for reproducibility.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and insightful comments, which highlight important aspects for strengthening our claims on the MCM-VG framework. We address each major comment point-by-point below, agreeing that additional analysis and experiments will improve the manuscript. We will incorporate these revisions in the next version.

read point-by-point responses

  1. Referee: [§3.2] §3.2 (Instance Rectification): The back-projection of VLM 2D masks to 3D point clouds is presented without any quantitative breakdown of correction success rate versus introduced error rate (e.g., from depth misalignment, occlusion, or VLM hallucination), which is load-bearing for the central claim that this module yields net-positive geometry corrections.

    Authors: We agree that a quantitative breakdown of correction success versus introduced errors is valuable to substantiate the net-positive impact of Instance Rectification. In the revised manuscript, we will add a dedicated analysis subsection reporting empirical success rates (via manual inspection on a sampled subset of ScanRefer cases), estimated error contributions from depth misalignment, occlusion, and VLM issues, and their net effect on final grounding accuracy. This will directly support the module's contribution to geometry corrections. revision: yes

  2. Referee: [§4] §4 (Experiments): No ablation studies are reported that isolate the contribution of Semantic Alignment, Instance Rectification, and Viewpoint Distillation individually, leaving open whether the 6.4% and 4.0% gains on ScanRefer derive from the proposed consistency mappings or from unstated tuning, baseline re-implementations, or other factors.

    Authors: We acknowledge that isolating each module's contribution is essential to attribute the reported gains specifically to the multiple consistent 2D-3D mappings. We will add detailed ablation studies in the revised experiments section, including performance on ScanRefer when disabling Semantic Alignment, Instance Rectification, or Viewpoint Distillation one at a time (and combinations), with comparisons to the full model and baselines. This will confirm the gains arise from the proposed mechanisms. revision: yes

  3. Referee: [§3.1] §3.1 (Semantic Alignment): The LLM-driven query parsing and coarse-to-fine matching lack analysis of category misalignment propagation through the pipeline, which directly affects the reliability of the final grounding when LLM parsing errors occur.

    Authors: We recognize the need for explicit analysis of how category misalignment errors propagate and impact final grounding reliability. In the revision, we will include an analysis in §3.1 (or a new subsection) that quantifies LLM parsing error rates on the benchmarks, examines propagation through the coarse-to-fine matching, and demonstrates mitigation effectiveness (e.g., via error injection experiments or statistics on mismatch correction success). This will address concerns about pipeline robustness. revision: yes

Circularity Check

0 steps flagged

No circularity: MCM-VG introduces independent algorithmic modules

full rationale

The paper proposes MCM-VG as a new framework that explicitly constructs Multiple Consistent 2D-3D Mappings via three distinct modules (Semantic Alignment with LLM parsing and coarse-to-fine matching, Instance Rectification by back-projecting VLM 2D segmentations, and Viewpoint Distillation for clustering camera directions). These steps are presented as algorithmic solutions to bottlenecks in open-vocabulary 3D proposals, with final grounding formulated as a multiple-choice VLM task on paired RGB and BEV prompts. Performance numbers (62.0% Acc@0.25, 53.6% Acc@0.5 on ScanRefer) are reported from benchmark evaluations rather than any fitted parameter or self-defined quantity. No equation, module, or claim reduces by construction to its own inputs, self-citations, or renamed priors; the derivation chain remains self-contained with external empirical content.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the practical effectiveness of the three modules; no new physical entities are postulated.

axioms (1)
  • domain assumption Pre-trained LLMs and VLMs can perform reliable query parsing, 2D segmentation, and multiple-choice reasoning on the given inputs.
    All three modules depend on the out-of-the-box capabilities of these models without additional fine-tuning described in the abstract.

pith-pipeline@v0.9.0 · 5648 in / 1369 out tokens · 67173 ms · 2026-05-07T13:43:01.199273+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 10 canonical work pages · 9 internal anchors

  1. [1]

    Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, and Leonidas Guibas. 2020. Referit3d: Neural listeners for fine-grained 3d object identification in real-world scenes. InEuropean Conference on Computer Vision. Springer, 422–440

    2020

  2. [2]

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al . 2025. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631(2025)

    work page internal anchor Pith review arXiv 2025

  3. [3]

    ByteDance. 2025. Seed1.6 Tech Introduction. https://seed.bytedance.com/en/ seed1_6. Accessed: September 2025

    2025

  4. [4]

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, An- drew Huang, et al. 2025. Sam 3: Segment anything with concepts.arXiv preprint arXiv:2511.16719(2025)

    work page internal anchor Pith review arXiv 2025

  5. [5]

    Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. 2020. Scanrefer: 3d object localization in rgb-d scans using natural language. InEuropean Conference on Computer Vision. Springer, 202–221

    2020

  6. [6]

    Liang Geng, Jianqin Yin, Gang Chen, and Qingxuan Jia. 2025. Pseudo-EV: Enhanc- ing 3D Visual Grounding with Pseudo Embodied Viewpoint.IEEE Transactions on Circuits and Systems for Video Technology(2025)

    2025

  7. [7]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. 2025. Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    work page internal anchor Pith review arXiv 2025

  8. [8]

    Wenxuan Guo, Xiuwei Xu, Ziwei Wang, Jianjiang Feng, Jie Zhou, and Jiwen Lu. 2025. Text-guided sparse voxel pruning for efficient 3d visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3666–3675

    2025

  9. [9]

    Zoey Guo, Yiwen Tang, Ray Zhang, Dong Wang, Zhigang Wang, Bin Zhao, and Xuelong Li. 2023. Viewrefer: Grasp the multi-view knowledge for 3d visual grounding. InProceedings of the IEEE/CVF International Conference on Computer Vision. 15372–15383

    2023

  10. [10]

    Pin-Hao Huang, Han-Hung Lee, Hwann-Tzong Chen, and Tyng-Luh Liu. 2021. Text-guided graph neural networks for referring 3d instance segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 1610–1618

    2021

  11. [11]

    Shijia Huang, Yilun Chen, Jiaya Jia, and Liwei Wang. 2022. Multi-view transformer for 3d visual grounding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 15524–15533

    2022

  12. [12]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card.arXiv preprint arXiv:2410.21276(2024)

    work page internal anchor Pith review arXiv 2024

  13. [13]

    Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. 2023. LERF: Language Embedded Radiance Fields. InProceedings of the IEEE/CVF International Conference on Computer Vision

    2023

  14. [14]

    Rong Li, Shijie Li, Lingdong Kong, Xulei Yang, and Junwei Liang. 2025. Seeground: See and ground for zero-shot open-vocabulary 3d visual grounding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3707– 3717

    2025

  15. [15]

    Jiawen Lin, Shiran Bian, Yihang Zhu, Wenbin Tan, Yachao Zhang, Yuan Xie, and Yanyun Qu. 2025. SeqVLM: Proposal-Guided Multi-View Sequences Reasoning via VLM for Zero-Shot 3D Visual Grounding. InProceedings of the 33rd ACM International Conference on Multimedia. 3094–3103

    2025

  16. [16]

    Junyu Luo, Jiahui Fu, Xianghao Kong, Chen Gao, Haibing Ren, Hao Shen, Huaxia Xia, and Si Liu. 2022. 3d-sps: Single-stage 3d visual grounding via referred point progressive selection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16454–16463

    2022

  17. [17]

    Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. 2023. Openscene: 3d scene understanding with open vocabularies. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 815–824

    2023

  18. [18]

    Zhipeng Qian, Yiwei Ma, Zhekai Lin, Jiayi Ji, Xiawu Zheng, Xiaoshuai Sun, and Rongrong Ji. 2024. Multi-branch collaborative learning network for 3d visual grounding. InEuropean Conference on Computer Vision. Springer, 381–398

    2024

  19. [19]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning. PmLR, 8748–8763

    2021

  20. [20]

    Jonas Schult, Francis Engelmann, Alexander Hermans, Or Litany, Siyu Tang, and Bastian Leibe. 2023. Mask3d: Mask transformer for 3d semantic instance segmentation. In2023 IEEE International Conference on Robotics and Automation. IEEE, 8216–8223

    2023

  21. [21]

    Xiangxi Shi, Zhonghua Wu, and Stefan Lee. 2024. Aware visual grounding in 3d scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14056–14065

    2024

  22. [22]

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al

  23. [23]

    OpenAI GPT-5 System Card

    Openai gpt-5 system card.arXiv preprint arXiv:2601.03267(2025). Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  24. [24]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805(2023)

    work page internal anchor Pith review arXiv 2023

  25. [25]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

    work page internal anchor Pith review arXiv 2023

  26. [26]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191(2024)

    work page internal anchor Pith review arXiv 2024

  27. [27]

    Xinyi Wang, Na Zhao, Zhiyuan Han, Dan Guo, and Xun Yang. 2025. Augre- fer: Advancing 3d visual grounding via cross-modal augmentation and spatial relation-based referring. InProceedings of the AAAI Conference on Artificial Intel- ligence, Vol. 39. 8006–8014

    2025

  28. [28]

    Yanmin Wu, Xinhua Cheng, Renrui Zhang, Zesen Cheng, and Jian Zhang. 2023. Eda: Explicit text-decoupling and dense alignment for 3d visual grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 19231–19242

    2023

  29. [29]

    Runsen Xu, Zhiwei Huang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. 2024. Vlm-grounder: A vlm agent for zero-shot 3d visual grounding.arXiv preprint arXiv:2410.13860(2024)

    work page arXiv 2024

  30. [30]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

    work page internal anchor Pith review arXiv 2025

  31. [31]

    Jianing Yang, Xuweiyi Chen, Shengyi Qian, Nikhil Madaan, Madhavan Iyengar, David F Fouhey, and Joyce Chai. 2024. Llm-grounder: Open-vocabulary 3d visual grounding with large language model as an agent. In2024 IEEE International Conference on Robotics and Automation. IEEE, 7694–7701

    2024

  32. [32]

    Zhengyuan Yang, Songyang Zhang, Liwei Wang, and Jiebo Luo. 2021. Sat: 2d semantics assisted training for 3d visual grounding. InProceedings of the IEEE/CVF International Conference on Computer Vision. 1856–1866

    2021

  33. [33]

    Zhihao Yuan, Jinke Ren, Chun-Mei Feng, Hengshuang Zhao, Shuguang Cui, and Zhen Li. 2024. Visual programming for zero-shot open-vocabulary 3d visual grounding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20623–20633

    2024

  34. [34]

    Zhihao Yuan, Xu Yan, Yinghong Liao, Ruimao Zhang, Sheng Wang, Zhen Li, and Shuguang Cui. 2021. Instancerefer: Cooperative holistic understanding for visual grounding on point clouds through instance multi-level contextual referring. InProceedings of the IEEE/CVF International Conference on Computer Vision. 1791–1800

    2021

  35. [35]

    Taolin Zhang, Sunan He, Tao Dai, Zhi Wang, Bin Chen, and Shu-Tao Xia. 2024. Vision-language pre-training with object contrastive learning for 3d scene under- standing. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 7296–7304

    2024

  36. [36]

    positive/negative

    Lichen Zhao, Daigang Cai, Lu Sheng, and Dong Xu. 2021. 3dvg-transformer: Relation modeling for visual grounding on point clouds. InProceedings of the IEEE/CVF International Conference on Computer Vision. 2928–2937. A Overview In this supplementary material, we provide comprehensive techni- cal details, extended analyses, and additional qualitative results...

    2021

  37. [37]

    Extract the single most accurate target object category from the query

    Target category A. Extract the single most accurate target object category from the query. B. The target category must appear exactly in the Object category list

  38. [38]

    to the left of the lamp

    Spatial references A. Extract object categories that are mentioned only as spatial reference objects in the query (e.g., “to the left of the lamp”). B. Store them as a list under spatial_refs. C. Do not treat spatial reference objects as target objects

  39. [39]

    Select exactly 10 unique categories from the Object category list

    Top category candidates A. Select exactly 10 unique categories from the Object category list. B. Do not include duplicate categories, even if they appear multiple times in the list. C. Rank categories from highest to lowest relevance based on: - Semantic similarity to the target object - Common furniture taxonomy (parent / sibling categories) - Typical us...

  40. [40]

    Output only a valid JSON object

  41. [41]

    Use exactly the following keys and structure: - query (string) - target_category (string) - spatial_refs (list of strings) - top_categories (list of 10 strings, ordered by relevance)

  42. [42]

    query":

    Do not add explanations, comments, or extra fields. Output JSON Template: { "query": "<original query sentence>", "target_category": "<single category from Object category list>", "spatial_refs": ["<category_1>", "<category_2>", "... "], "top_categories": [ "<category_1>", "<category_2>", "<category_3>", "<category_4>", "<category_5>", "<category_6>", "<c...

  43. [43]

    Positive Point 1 [x1, y1]: The GEOMETRIC CENTER of the target

  44. [44]

    Positive Point 2 [x2, y2]: A corner or boundary point (e.g., top-left) that defines the scale of the target

  45. [45]

    Presence

    Negative Point [x3, y3]: A point clearly OUTSIDE the target’s boundary, typically on a nearby distracting object or background, used to clarify the target’s limits. - Coordinates: Use NORMALIZED COORDINATES [0-1000] (where [0,0] is top-left and [1000,1000] is bottom-right). Multiple Consistent 2D-3D Mappings for Robust Zero-Shot 3D Visual Grounding Confer...

    2018

  46. [46]

    - The right side displays the BEV (Bird’s-Eye View) map of the room, which represents the top-down spatial layout of the room

    Each input image is a composite: - The left side displays perspective camera views from a sequence, where the target object is highlighted by a red rectangle. - The right side displays the BEV (Bird’s-Eye View) map of the room, which represents the top-down spatial layout of the room

  47. [47]

    Understanding the BEV Map: The red dots and arrows in the BEV map represent the camera’s spatial position and its viewing direction (orientation) at the moment the left image was captured

  48. [48]

    Spatial Correspondence: The red marker or arrow in the BEV map on the right corresponds to the specific object shown in the red boxes on the left

  49. [49]

    ‘ ***. {

    Your Task: You must combine the visual appearance (from the left sub-images) with the spatial context (from the right BEV map, e.g., location in the room, proximity to walls or other furniture) to make your selection. Your response should be in the following format, and it should not include code block markers such as “‘ ***. { "process": "Explain the pro...

  50. [50]

    Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al

    Focus more on the **Object Category** (e.g., is it a trash can?) and **Color/Texture**. Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al

    2018

  51. [51]

    Focus on the **Spatial Location** (e.g., right of the door). 3. Be tolerant of minor shape discrepancies. If there is an image that is a STRONG match for category and location, select it even if the shape isn’t perfect. If you still strictly believe none match, return -1