Pith. sign in

REVIEW 4 major objections 5 minor 28 references

TinyGiantVLM: A Lightweight Vision-Language Architecture for Spatial Reasoning under Resource Constraints

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A compact 64M-parameter vision-language model achieves a top-five result on a warehouse spatial-reasoning benchmark.

desk verdict A real 5th-place competition result from a genuinely light model, but the paper's signature MoE module was never in the official submission and its only evidence is a garbled validation-only ablation. read the letter →

arxiv 2508.17595 v1 pith:QBV2GR33 submitted 2025-08-25 cs.CV

Vinh-Thuan Ly , Hoang M. Truong , Xuan-Huong Nguyen This is my paper · ORCID
classification cs.CV
keywords vision-languagemodelspatialreasoningRGB-DfusionregioninjectionMixture-of-Expertstwo-phasetrainingwarehousesceneunderstandingresource-constraineddeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TinyGiantVLM is a compact vision-language model that reasons about physical space in warehouse scenes from RGB images and depth maps, using only 64 million trainable parameters in its official submission. The paper claims that with a dual-branch design, which fuses global and region-level features from both modalities, injects region vectors into question tokens, and trains first on free-form answers then on normalized ones, this small model places fifth on a warehouse spatial-intelligence benchmark. If the claim holds, heavy spatial Q&A does not require billion-parameter language models, and modest hardware can support a useful class of industrial scene-understanding tasks.

What carries the argument

The load-bearing mechanism is the region-injection path: RGB and depth images are encoded by two frozen vision transformers, one for RGB and one for depth, and each annotated region is summarized by mean-pooling the patch embeddings inside its mask. Those region vectors are projected and injected directly into a small text-to-text transformer encoder by replacing the placeholder tokens that stand for <mask> in the question, so the language model contextualizes visual regions in place. Cross-attention then lets each region embedding attend to the global scene vector, and the resulting representations are re-injected into encoder outputs before the decoder generates an answer. A sparsely-gated Mixture-of-Experts layer with four task-specialized experts and distance-based Laplace gating is optionally inserted to route region tokens; the two-phase curriculum, free-form answers first and normalized answers second, adapts the decoder from exploratory language to the exact answer format required by the evaluation.

What would settle it

Retrain the same two-phase pipeline with the discarded 120,000 distance samples included, or subsampled randomly instead of dropped from the front, and compare distance accuracy: if it rises substantially above 50.26%, the reported gap reflects data curation rather than monocular-depth limits.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that a 64M-parameter, non-MoE RGB-D sequence-to-sequence model can place 5th on the warehouse spatial-intelligence leaderboard with a score of 66.89%, while an 80M-parameter MoE variant and longer training raise validation accuracy to 72.52% and 88.52% respectively. The empirical centerpiece is the ablation table showing that both ingredients matter: free-form pretraining alone gives 25.59%, normalized fine-tuning alone gives 63.65%, and their combination reaches 65.09%; adding MoE to both phases yields 72.52%. The paper also reports per-task accuracies, counting at 83.87%, left/right relations at 98.40%, distance estimation at 50.26%, and MCQ grounding at 35.01%, attributing the weak distance result to the limits of monocular depth and the weak MCQ result to region disambiguation difficulty.

Load-bearing premise

The reported task accuracies assume that dropping the first 120,000 distance-estimation samples from the training set does not systematically change the difficulty of the distance task.

Editorial extensions

If this is right

  • A 64M-parameter model without MoE reaches 5th place (66.89%) on the benchmark, showing that warehouse spatial reasoning does not require a billion-parameter LLM.
  • Two-phase training, free-form then normalized answers, beats either phase alone: 65.09% versus 25.59% for Phase 1 only and 63.65% for Phase 2 only without MoE.
  • Activating the MoE fusion layer with four task experts and k=2 raises validation accuracy to 72.52% when combined with both phases, at a cost of about 16M extra parameters.
  • Extending Phase 2 training from 10 to 25 epochs lifts the non-MoE two-phase model from 65.09% to 88.52% on validation, indicating the official result was compute-limited rather than architecture-limited.
  • Per-task, the model is strong at counting (83.87%) and left/right relations (98.40%), weaker at distance estimation (50.26%) and MCQ grounding (35.01%).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 25-epoch validation gain transfers to the held-out test, the architecture's capacity is not the limiting factor; the official 5th-place score likely understates what the method can do with a larger compute budget.
  • The distance-estimation weakness is confounded by the deliberate removal of 120,000 distance samples; a random-downsampling control would separate data-curation effects from true monocular-depth limits.
  • Region features are computed as averages of patch embeddings inside masks; the low MCQ-grounding accuracy (35.01%) suggests that average pooling discards spatial layout, so attention-based or positional aggregation over mask patches is a natural next test.
  • The MoE module was not active in the official submission; the validation gain from MoE (72.52%) needs a held-out confirmation before being treated as a real improvement.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents TinyGiantVLM, a lightweight vision-language architecture for spatial reasoning in warehouse scenes, evaluated on Track 3 of the AI City Challenge 2025. The proposed system uses dual RGB/depth feature extraction at global and region levels, region-aware feature injection into a T5-small encoder, cross-attention fusion, and a Mixture-of-Experts (MoE) fusion layer with Laplace gating. Training is done in two phases: free-form answer generation followed by fine-tuning on normalized answers. The paper reports that the 64M-parameter non-MoE variant ranked 5th on the public leaderboard with a score of 66.8861, and that post-deadline ablation studies on the validation set show gains from the MoE module (up to 72.52%) and from longer training (up to 88.52%). The central tension is that the official leaderboard submission did not use the MoE module, so the externally verified result concerns only the non-MoE architecture, while the paper's title and abstract foreground the MoE contribution.

Significance. If the empirical claims are substantiated, the paper demonstrates that a compact RGB-D VLM with region-level feature injection and a two-phase training curriculum can be competitive in an industrial spatial-reasoning benchmark under severe compute constraints (a single P100 GPU). The externally supported 5th-place leaderboard result is a concrete strength, as are the practical engineering choices of feature caching and lightweight fusion. However, the most distinctive architectural component, the MoE fusion layer, is not part of the official submission, and its only support is a validation-only, internally garbled ablation with no matched-compute baseline, no repeated seeds, and no released code. The paper is therefore best viewed as a promising systems/challenge report whose central architectural claim needs to be either substantially re-framed or verified with a reproducible, controlled study.

major comments (4)
  1. [Sec. 3.2 and Fig. 2] The caption of Figure 2 states that the MoE module 'was not activated in the final evaluation due to implementation issues,' but the abstract and introduction present MoE as a component of the architecture that achieved 5th place. The externally supported leaderboard result therefore validates only the non-MoE dual-branch, region-injection, two-phase system. Please re-state the contributions so that the validated system and the proposed-but-unverified MoE extension are clearly separated in the abstract, introduction, and conclusion.
  2. [Table 1] The check/cross markers in Table 1 are corrupted (rows display '7' and '3' rather than readable indicators), and the table cannot be deciphered without the prose. More substantively, the claimed MoE gain (72.52% versus 65.09%) is a validation-only comparison with no error bars, no repeated seeds, and no matched compute or epoch budget. The post-deadline 25-epoch non-MoE result (88.52%) shows that training duration alone can produce a much larger improvement than the reported MoE effect, so the ablation as presented does not isolate the MoE contribution. Please repair the table and provide a controlled comparison that matches training duration and compute across MoE and non-MoE configurations, ideally with multiple seeds.
  3. [Sec. 4.2] The decision to discard the first 120,000 samples from the distance category is not analyzed. Because these samples are taken from the beginning of the dataset rather than a random subsample, their distribution over scene configurations, difficulty levels, and question formats is unknown, and their removal can directly depress the distance-estimation accuracy (50.26%) that Section 4.4 attributes to 'inherent limitations of monocular depth and lack of explicit geometric supervision.' Please report the size and composition of the remaining distance subset and provide a sensitivity analysis, such as training on a random subsample of the same size or on the full distance set, to determine whether the reported task-level accuracy is an artifact of data truncation.
  4. [Secs. 4.3 and 4.4] The two-phase training comparison depends on a free-form-to-normalized conversion step that uses an external instruction-tuned model (Qwen3 1.7B), but this post-processing step is not described in the methodology and is not part of the reported architecture. Since the Phase 1 versus Phase 2 gap (25.59% versus 63.65%) may partly reflect the behavior and cost of this converter, please specify the conversion protocol, its error modes, and whether it was applied consistently across all ablation rows.
minor comments (5)
  1. [Eqs. (11)-(12)] The cross-attention formulation uses the global vector g as both key and value, which gives a single-vector attention context; please clarify the dimensions and state explicitly how multi-head attention is applied over the R region queries and what d denotes.
  2. [Sec. 4.2] The resource-constraint claim would be stronger if the paper reported wall-clock time per epoch and total training time on the P100 GPU, in addition to the model parameter count.
  3. [Sec. 4.1] Please provide a precise description of the PhysicalAI-Spatial-Intelligence-Warehouse dataset split used for Table 1, including whether the validation set is the same split used for the leaderboard and how the discarded distance samples interact with that split.
  4. [Abstract and Sec. 4.4] The abstract reports a score of 66.8861 while Section 4.4 writes '66.89%'; please use the same precision and formatting consistently.
  5. [Fig. 2] Figure 2 is low-resolution and many text labels are garbled or truncated; please regenerate the figure with readable typography so the architecture diagram is legible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the system is measured against an external leaderboard, and the MoE/two-phase claims are reported as direct validation ablations rather than as predictions forced by fitted inputs.

full rationale

This is an empirical system paper, not a derivation chain that could collapse into its inputs. The central performance claim is anchored to the external Track 3 leaderboard ('our 64M-parameter base model achieved 5th place on the leaderboard with a score of 66.8861'), which is an independent benchmark and therefore not a fitted quantity renamed as a prediction. The architectural components—CLIP-ViT, DPT, T5-small, region injection, cross-attention, and the Laplace-gated MoE layer—are borrowed with explicit citations; the Laplace gating is 'inspired by FuseMoE [4]' and adopted as a design choice, not derived from this paper's own outputs, so no ansatz is smuggled in as an external theorem. The ablation in Table 1 compares measured validation accuracies across configurations; despite the garbled glyphs in the markdown rendering ('7' vs '3'), the prose in Section 4.3 decodes the rows consistently, and the reported differences are direct measurements rather than constructions that force the conclusion. The in-scope limitation statement in the Figure 2 caption—'While this MoE module is part of our proposed design, it was not activated in the final evaluation due to implementation issues'—means the official leaderboard score validates only the non-MoE variant; this is a support gap for the MoE contribution, not circularity, because the paper explicitly labels the MoE gains as post-deadline validation ablations without presenting them as the official result. Likewise, the decision in Section 4.2 to 'discard the first 120,000 samples from the distance category' is a data-selection confound for interpreting the distance accuracy of 50.26%, but it is not a self-referential reduction. No fitted parameter is renamed as a prediction, no load-bearing claim rests on a same-author citation chain, and no claimed result reduces to its own definition or to the training data by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical result rests on a simulated benchmark, frozen visual backbones, a hand-chosen data discard, and hand-set epoch and MoE hyperparameters. No new physical entities are introduced. The two-phase curriculum is a paper-specific assumption validated only on one validation set.

free parameters (4)
  • Discarded distance samples = 120,000
    Section 4.2 removes the first 120,000 samples of the distance category from training to reduce compute. The choice of which and how many samples to drop is hand-chosen and unanalyzed; it can bias the distance estimation accuracy (50.26%) reported in Section 4.4.
  • Training epoch schedule = Phase 1: 1 epoch; Phase 2: 10 epochs; post-deadline: 25 epochs
    The 10-to-25 epoch increase changes the non-MoE two-phase validation score from 65.09% to 88.52%, so the epoch budget is a major uncontrolled variable in the ablation that compares architectures at 10 epochs.
  • MoE router settings = S=4 experts, k=2 top experts
    Section 3.2 defines four experts for the four question types and top-2 routing; these hand-set values determine whether the expert specialization claim holds.
  • Learning rate / weight decay / batch size = 5e-5, 1e-2, 32
    Standard but arbitrary hyperparameters; listed for completeness since the network is trained, not derived.
assumptions (4)
  • domain assumption The PhysicalAI-Spatial-Intelligence-Warehouse dataset, generated with NVIDIA Omniverse IsaacSim, is a valid proxy for real warehouse spatial reasoning.
    Section 4.1 introduces the dataset as simulated industrial layouts; the paper generalizes to 'industrial deployment' without sim-to-real validation.
  • domain assumption Frozen CLIP-ViT and DPT encoders provide features sufficiently rich for spatial reasoning from RGB and depth.
    Section 3.1 freezes the backbones and caches features; the paper does not compare with trainable encoders.
  • domain assumption Downsampling RLE region masks to the patch grid (16x16 RGB, 24x24 depth) and averaging patch embeddings preserves the object identity needed for MCQ grounding.
    Eqs. (2)-(3) average all patch features in the region; the low MCQ accuracy (35.01%) suggests this assumption is fragile.
  • ad hoc to paper A two-phase curriculum of free-form answers followed by normalized answers improves final task accuracy.
    This is the paper's own training claim, tested only on the validation set without error bars.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TinyGiantVLM: A Lightweight Vision-Language Architecture for Spatial Reasoning under Resource Constraints." pith.science (2026). https://pith.science/paper/QBV2GR33

@misc{pith2026250817595,
  author       = {Pith},
  title        = {Pith review of: TinyGiantVLM: A Lightweight Vision-Language Architecture for Spatial Reasoning under Resource Constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QBV2GR33}},
  note         = {Machine review of arXiv:2508.17595}
}
read the original abstract

Reasoning about fine-grained spatial relationships in warehouse-scale environments poses a significant challenge for existing vision-language models (VLMs), which often struggle to comprehend 3D layouts, object arrangements, and multimodal cues in real-world industrial settings. In this paper, we present TinyGiantVLM, a lightweight and modular two-stage framework designed for physical spatial reasoning, distinguishing itself from traditional geographic reasoning in complex logistics scenes. Our approach encodes both global and region-level features from RGB and depth modalities using pretrained visual backbones. To effectively handle the complexity of high-modality inputs and diverse question types, we incorporate a Mixture-of-Experts (MoE) fusion module, which dynamically combines spatial representations to support downstream reasoning tasks and improve convergence. Training is conducted in a two-phase strategy: the first phase focuses on generating free-form answers to enhance spatial reasoning ability, while the second phase uses normalized answers for evaluation. Evaluated on Track 3 of the AI City Challenge 2025, our 64M-parameter base model achieved 5th place on the leaderboard with a score of 66.8861, demonstrating strong performance in bridging visual perception and spatial understanding in industrial environments. We further present an 80M-parameter variant with expanded MoE capacity, which demonstrates improved performance on spatial reasoning tasks.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 18 canonical work pages

  1. [1]

    Spatialvlm: Endow- ing vision-language models with spatial reasoning capabili- ties

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endow- ing vision-language models with spatial reasoning capabili- ties. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 14455– 14465, 2024. 2

    work page 2024

  2. [2]

    Spatial- rgpt: Grounded spatial reasoning in vision-language models

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Rui- han Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatial- rgpt: Grounded spatial reasoning in vision-language models. In NeurIPS, 2024. 2

    work page 2024

  3. [3]

    Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understand- ing, generation, and instruction following

    Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xi- anzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xi- anzhi Li, Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understand- ing, generation, and instruction following. arXiv preprint arXiv:2309.00615, 2023. 2

    arXiv 2023

  4. [4]

    Fusemoe: Mixture-of-experts transformers for flexi- modal fusion

    Xing Han, Huy Nguyen, Carl Harris, Nhat Ho, and Suchi Saria. Fusemoe: Mixture-of-experts transformers for flexi- modal fusion. In Advances in Neural Information Processing Systems, pages 67850–67900. Curran Associates, Inc., 2024. 2, 4, 5

    work page 2024

  5. [5]

    Multiply: A multisensory object- centric embodied large language model in 3d world

    Yining Hong, Zishuo Zheng, Peihao Chen, Yian Wang, Jun- yan Li, and Chuang Gan. Multiply: A multisensory object- centric embodied large language model in 3d world. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26406–26416, 2024. 2

    work page 2024

  6. [6]

    Chat-scene: Bridging 3d scene and large language models with object identifiers

    Haifeng Huang, Yilun Chen, Zehan Wang, Rongjie Huang, Runsen Xu, Tai Wang, Luping Liu, Xize Cheng, Yang Zhao, Jiangmiao Pang, et al. Chat-scene: Bridging 3d scene and large language models with object identifiers. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. 2

    work page 2024

  7. [7]

    Jordan and Robert A

    Michael I. Jordan and Robert A. Jacobs. Hierarchical mix- tures of experts and the em algorithm. Neural Computation, 6(2):181–214, 1994. 2

    work page 1994

  8. [8]

    Chang, and Manolis Savva

    Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X. Chang, and Manolis Savva. Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal naviga- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Patte...

    work page 2024

Show all 28 references
  1. [9]

    Sparse mixture- of-experts are domain generalizable learners, 2023

    Bo Li, Yifei Shen, Jingkang Yang, Yezhen Wang, Jiawei Ren, Tong Che, Jun Zhang, and Ziwei Liu. Sparse mixture- of-experts are domain generalizable learners, 2023. 2, 4

    2023

  2. [10]

    Shapellm: Universal 3d object understanding for embodied interaction

    Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. In European Conference on Computer Vision , pages 214–

  3. [11]

    Gpt4point: A unified framework for point-language understanding and generation

    Zhangyang Qi, Ye Fang, Zeyi Sun, Xiaoyang Wu, Tong Wu, Jiaqi Wang, Dahua Lin, and Hengshuang Zhao. Gpt4point: A unified framework for point-language understanding and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26417–...

    2024

  4. [12]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  5. [13]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learn- ing Research, 21(140):1–67, 2020. 2, 4

    2020

  6. [14]

    Vi- sion transformers for dense prediction, 2021

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction, 2021. 2, 3

    2021

  7. [15]

    Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer. arXiv preprint arXiv:1701.06538, 2017. 2, 4

    2017 arXiv

  8. [16]

    Minigpt-3d: Efficiently aligning 3d point clouds with large language models using 2d priors

    Yuan Tang, Xu Han, Xianzhi Li, Qiao Yu, Yixue Hao, Long Hu, and Min Chen. Minigpt-3d: Efficiently aligning 3d point clouds with large language models using 2d priors. In Pro- ceedings of the 32nd ACM International Conference on Mul- timedia, pages 6617–6626, 2024. 2

    2024

  9. [17]

    Zheng Tang, Shuo Wang, David C. Anastasiu, Ming- Ching Chang, Anuj Sharma, Quan Kong, Norimasa Ko- bori, Munkhjargal Gochoo, Ganzorig Batnasan, Munkh- Erdene Otgonbold, Fady Alnajjar, Jun-Wei Hsieh, Tomasz Kornuta, Xiaolong Li, Yilin Zhao, Han Zhang, Subhashree Radhakrishnan, ...

    2025

  10. [18]

    Chat-3d: Data-efficiently tuning large language model for universal dialogue of 3d scenes

    Zehan Wang, Haifeng Huang, Yang Zhao, Ziang Zhang, and Zhou Zhao. Chat-3d: Data-efficiently tuning large language model for universal dialogue of 3d scenes. arXiv preprint arXiv:2308.08769, 2023. 2

    2023 arXiv

  11. [19]

    Pointllm: Empowering large language models to understand point clouds

    Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiang- miao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds. In European Conference on Computer Vision , pages 131–147. Springer,

  12. [20]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chen- gen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, J...

    2025 arXiv

  13. [21]

    RoboPoint: A vision-language model for spatial affordance prediction for robotics

    Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousa- vian, and Dieter Fox. RoboPoint: A vision-language model for spatial affordance prediction for robotics. In Conference on Robot Learning (CoRL), 2024. 1

    2024

  14. [22]

    CounterCurate: Enhancing physical and semantic visio- linguistic compositional reasoning via counterfactual exam- ples

    Jianrui Zhang, Mu Cai, Tengyang Xie, and Yong Jae Lee. CounterCurate: Enhancing physical and semantic visio- linguistic compositional reasoning via counterfactual exam- ples. In Findings of the Association for Computational Lin- guistics: ACL 2024 , pages 15481–15495, Bangkok,...

    2024

  15. [23]

    Pointclip: Point cloud understanding by clip

    Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xu- peng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. Pointclip: Point cloud understanding by clip. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8552–8562, 2022. 2

    2022

  16. [24]

    Agent3d-zero: An agent for zero-shot 3d understanding

    Sha Zhang, Di Huang, Jiajun Deng, Shixiang Tang, Wanli Ouyang, Tong He, and Yanyong Zhang. Agent3d-zero: An agent for zero-shot 3d understanding. In European Confer- ence on Computer Vision , pages 186–202. Springer, 2024. 2

    2024

  17. [25]

    Uni3d: Exploring unified 3d representation at scale

    Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. arXiv preprint arXiv:2310.06773,

    arXiv

  18. [26]

    Scanreason: Empowering 3d visual grounding with reasoning capabilities

    Chenming Zhu, Tai Wang, Wenwei Zhang, Kai Chen, and Xihui Liu. Scanreason: Empowering 3d visual grounding with reasoning capabilities. In European Conference on Computer Vision, pages 151–168. Springer, 2024. 2

    2024

  19. [27]

    Llava-3d: A simple yet effective pathway to empowering lmms with 3d-awareness

    Chenming Zhu, Tai Wang, Wenwei Zhang, Jiangmiao Pang, and Xihui Liu. Llava-3d: A simple yet effective pathway to empowering lmms with 3d-awareness. arXiv preprint arXiv:2409.18125, 2024. 2

    2024 arXiv

  20. [28]

    Point- clip v2: Prompting clip and gpt for powerful 3d open-world learning

    Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyu Guo, Ziyao Zeng, Zipeng Qin, Shanghang Zhang, and Peng Gao. Point- clip v2: Prompting clip and gpt for powerful 3d open-world learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2639–2650, 2023. 2

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.