Pith. sign in

REVIEW 2 major objections 6 minor 73 references

Describe, Adapt and Combine: Empowering CLIP Encoders for Open-set 3D Object Retrieval

T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Unseen 3D object retrieval jumps ~10 mAP with CLIP plus text

desk verdict Good simple idea and thorough ablations, but the headline mAP gain doesn't match Table 1 and the per-dataset α looks test-set tuned. read the letter →

arxiv 2507.21489 v1 pith:UUGHUGWH submitted 2025-07-29 cs.CV

Zhichuan Wang , Yang Zhou , Zhe Liu , Rui Yu , Song Bai , Yulong Wang , Xinwei He , Xiang Bai This is my paper · ORCID
classification cs.CV
keywords open-set3DobjectretrievalCLIPmulti-modallargelanguagemodellow-rankadaptationmulti-viewimageszero-shotcross-dataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-set 3D object retrieval asks a system to fetch 3D models of categories it never saw during training, and most prior methods need voxels, point clouds, and test-data supervision to do this. This paper proposes DAC, which uses only multi-view images, a frozen CLIP encoder, and an off-the-shelf multimodal language model that writes one-sentence descriptions of the seen categories during training and of each query object at inference. A lightweight additive-bias LoRA adapts CLIP to the rendered views, and the visual and textual embeddings are combined by weighted addition followed by tanh normalization. The authors report that DAC beats the previous best method by an average of 10.01 mAP across four open-set 3DOR benchmarks, and that it also improves cross-dataset, single-image, and depth-based retrieval.

What carries the argument

The load-bearing mechanism is the pairing of CLIP's aligned image-text embedding space with an MLLM that writes descriptions, plus a modified parameter-efficient adapter. AB-LoRA extends LoRA by adding a trainable bias vector $\Phi$ to each adapted linear layer's output, $o = W_o z + \gamma B A z + \Phi$; because LoRA's update accumulates information from seen-category inputs, the bias is intended to prevent that update from overfitting the seen categories and to preserve generalization to unseen ones. During training the MLLM generates one description per category, which becomes the text classifier in a cross-entropy contrastive loss over pooled multi-view features; at inference it generates one description per object, whose CLIP text embedding is added to the visual embedding with weight $\alpha$ and passed through tanh to form the final 3D descriptor.

What would settle it

Re-run the main comparison with a single fixed $\alpha$ for all four datasets, or choose $\alpha$ on a held-out validation split drawn from the probe/gallery distribution before evaluation; if the average margin over HGM2R falls below the claimed 10.01 mAP, the headline result depends on per-benchmark test-set tuning.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a generative multimodal large language model and a discriminative contrastive model can be combined into a strong open-set 3D retriever without any 3D backbone and without touching test data. The method works in three steps: an MLLM supplies category-level descriptions that align with CLIP's training objective, an Additive-Bias Low-Rank adaptation (AB-LoRA) fine-tunes the attention weights of CLIP's visual and textual encoders to close the gap between natural images and multi-view projections while resisting overfitting to seen classes, and a simple elementwise fusion $h = \tanh(g + \alpha f_t)$ joins pooled multi-view features with text embeddings. Trained only on seen categories and evaluated on unseen ones, DAC reports 58.70 to 70.74 mAP across the four benchmarks with a ViT-L/14 backbone, surpassing HGM2R and other multi-modal baselines while using considerably less input information.

Load-bearing premise

The reported gains assume the per-dataset fusion weights $\alpha$ listed as 'optimal' were chosen without looking at test retrieval labels; if they were tuned on the test set, the headline margins, especially the roughly 20 mAP gain on OS-NTU-core, would be inflated by test-set fitting.

Editorial extensions

If this is right

  • A 3D retriever can be built without point-cloud or voxel encoders: projecting an object into 24 images and reading its category descriptions is enough to beat multi-modal baselines.
  • The same recipe transfers to cross-dataset retrieval, single-image retrieval, and zero-shot depth-image retrieval, suggesting the CLIP-plus-MLLM synergy is not limited to one dataset or modality.
  • The additive bias in AB-LoRA is what lets the model keep generalization to unseen categories; removing it costs about 2.55 mAP on OS-MN40-core.
  • If the per-dataset fusion weight $\alpha$ is a genuine design choice rather than a test-set fit, DAC's margin over HGM2R holds without any test data during training, which makes the method practical for repositories that grow over time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If $\alpha$ were chosen by a principled rule, such as predicting it from the MLLM's confidence or fixing it across all four benchmarks, the method would be more likely to transfer to a new dataset without per-benchmark tuning; the current table of optimal $\alpha$ values leaves that question open.
  • The same Describe-Adapt-Combine pattern could be applied to other retrieval domains where CLIP has a strong prior and an MLLM can verbalize unseen classes, such as sketch retrieval or fine-grained species identification, though the paper does not test these.
  • Because stronger MLLMs (up to Qwen2.5-VL-7B) monotonically improve DAC's mAP, part of the claimed gap over prior methods is a bet on continuing MLLM progress rather than on the 3D representation itself.
  • A testable extension: replacing the scalar $\alpha$ with a per-query weight, based on how confident the MLLM's description is, could improve retrieval on hard pairs like wardrobe versus bookshelf that currently fail.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes DAC, a framework for open-set 3D object retrieval that uses only multi-view images. DAC first employs an off-the-shelf MLLM (InternVL) to generate category-level descriptions for training and object-level descriptions at inference, then adapts a frozen CLIP encoder with a proposed Additive-Bias Low-Rank Adaptation (AB-LoRA), and finally combines visual and textual CLIP embeddings via the weighted fusion h = tanh(g + α f_t). Experiments on four open-set 3DOR datasets report an average improvement of +10.01% mAP over prior art in the open-set setup, together with cross-dataset, single-image, and depth-image retrieval experiments.

Significance. If the reported numbers hold, DAC is a simple and effective new state of the art for open-set 3DOR, notable for relying only on multi-view images and for not using test data during training, in contrast to HGM2R. The paper is thorough in its ablations (LoRA rank, fusion scheme, normalization, prompts, MLLM choice, view numbers), releases code, and explicitly discusses limitations in Sec. 4.6. The central caveat is the evaluation protocol for the per-dataset fusion weight α and the absence of variance estimates; because the headline gain is purely quantitative, these issues directly affect the strength of the state-of-the-art claim.

major comments (2)
  1. [Appendix B.2 (Table 8) and Eq. (6)] The fusion weight α in Eq. (6) is reported as an 'optimal' value per dataset and per backbone in Table 8, with the text stating 'We set these values as our default configurations.' No held-out validation split is described for choosing these eight values, and Figure 5 sweeps α on OS-MN40-core and reports the resulting test-set mAP. Consequently, the headline improvement of +10.01% mAP in the Abstract, which rests on the open-set rows of Table 1, may be inflated by test-set fitting. The concern is concrete: the largest single gain (OS-NTU-core, ViT-L/14, +20.95 mAP over HGM2R) uses α=0.3, and the cross-dataset and single-image results in Tables 5 and 6 also rely on per-dataset α choices. Please either specify a validation-based selection protocol that fixes α independently of the test set, or report sensitivity with a fixed α across all datasets and show that the ranking over prior methods is preserved.
  2. [Tables 1, 5, and 6] All retrieval results are reported as single runs without error bars or multiple seeds. The Abstract uses the word 'significantly surpasses,' but with one run per configuration there is no estimate of variance, and the claimed margins (e.g., +12.31% mAP in Table 5) could be within run-to-run variability. Please provide mean and standard deviation over at least three independent training runs for the main comparisons, or at least for Table 1 and Table 8, and state the number of seeds used for every reported number.
minor comments (6)
  1. [Sec. 3.1 and Sec. 4.2] There are several typos: 'pretrainned' should be 'pretrained', 'Multi-model Large Language Models' should be 'Multi-modal', 'siginantly' should be 'significantly', and 'remarkablely' should be 'remarkably'.
  2. [Appendix B.2 and Appendix G] Figures 5 and 8 appear in the submitted PDF as raw embedded paths (e.g., '/uni00000013/...') rather than rendered plots; the figures need to be regenerated before publication.
  3. [Eq. (6) and Table 8] Please clarify whether α is applied after normalizing g and f_t individually, and state the search grid and step size used to arrive at the 'optimal' α values in Table 8.
  4. [Table 1 and Sec. 4.2] The 'Zero-shot Setup' should be defined explicitly; the DAC rows in this block use MLLM-generated descriptions of the query object, so the setup is not identical to prior zero-shot methods that do not use query-specific text.
  5. [Sec. 3.2] The gradient derivation and the statement that adding Φ 'breaks the tight linkage' is heuristic; Table 3 provides empirical support, but the text should not imply a formal proof of why the bias improves unseen-category generalization.
  6. [Appendix D] The new ZS-Objaverse-Core benchmark should specify the number of categories and the exact 20%/80% split protocol more precisely, since current wording only says 'each category' without stating the number of categories used.

Circularity Check

1 steps flagged · score 6.0 of 10

Reported gains rest on per-dataset fusion weight α selected on the test split, so part of the claimed +10.01% mAP improvement is fitted rather than predicted.

  1. fitted input called prediction [Appendix B.2, Table 8, Eq. (6)]
    "h=tanh(g+α f_t)... where α∈[0,1] is a weighting factor... To study its effects, we adjust the fusion ratio α within the range of 0 to 1 and conduct experiments on OS-MN40-core... In Table 8, we further provide the optimal values for α across all the datasets... We set these values as our default configurations for our experiments."

    The final descriptor is h=tanh(g+α f_t) (Eq. 6), and retrieval metrics are computed on the test probe/gallery sets. For each dataset and backbone, α is selected as the value that maximizes those same test metrics ('optimal values', Table 8), with no held-out validation split described. The reported mAP/NDCG/ANMRR values are therefore optimized with respect to α on the evaluation set; the headline +10.01% mAP average and per-dataset gains (e.g., 65.83% on OS-NTU with α=0.3) are not predictions from a fixed protocol but partly a fit to the test labels. Because the compared baselines are not given the same per-dataset test-set tuning, the margin is inflated.

full rationale

DAC is an empirical pipeline, not a formal derivation, and most of its components are not circular: CLIP and InternVL are external pretrained models; descriptions for unseen objects are generated at inference from test views without using category labels; AB-LoRA is trained only on seen categories via Eq. (5); and the comparisons against HGM2R and other baselines are external benchmarks. No load-bearing self-citation or imported uniqueness argument was found. The one substantive circular element is the per-dataset fusion weight α: Appendix B.2 reports 'optimal α values across all the datasets' and sets them as defaults, which means the reported test-set numbers are partially constructed by test-set fitting rather than being out-of-the-box predictions. This directly affects the central quantitative claim of a +10.01% mAP average improvement, so the finding is partial but load-bearing.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's central claim rests on the effectiveness of CLIP and MLLM, not on new physical entities. The main fitted parameters are the fusion weight alpha and standard training hyperparameters. The key assumptions are about the sufficiency of multi-view projections and the transferability of pre-trained models.

free parameters (4)
  • fusion weight alpha = 0.1 to 0.85 depending on dataset and backbone (Table 8)
    Balances visual and text features in the final descriptor; selected per dataset/backbone to maximize performance.
  • LoRA rank r = 8
    Chosen via ablation on OS-MN40-core (Table 7).
  • dropout probability = 0.25
    Used to regularize LoRA input; standard hyperparameter.
  • learning rate = 2e-4
    SGD optimizer; standard hyperparameter.
assumptions (4)
  • domain assumption 24 multi-view gray-scale images capture sufficient 3D shape information for retrieval
    The paper projects each 3D object into 24 views following HGM2R (Sec. 4.1).
  • domain assumption CLIP's pre-trained image-text embedding space transfers to synthetic multi-view renderings
    The method adapts CLIP with LoRA but assumes the base CLIP features are a strong starting point (Sec. 3.2).
  • domain assumption MLLM-generated descriptions are accurate and complementary to visual features
    The method relies on InternVL to produce useful descriptions of seen and unseen objects (Sec. 3.1).
  • domain assumption The training and retrieval categories are disjoint in the open-set protocol
    Defined in the problem setup (Sec. 3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Describe, Adapt and Combine: Empowering CLIP Encoders for Open-set 3D Object Retrieval." pith.science (2026). https://pith.science/paper/UUGHUGWH

@misc{pith2026250721489,
  author       = {Pith},
  title        = {Pith review of: Describe, Adapt and Combine: Empowering CLIP Encoders for Open-set 3D Object Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UUGHUGWH}},
  note         = {Machine review of arXiv:2507.21489}
}
read the original abstract

Open-set 3D object retrieval (3DOR) is an emerging task aiming to retrieve 3D objects of unseen categories beyond the training set. Existing methods typically utilize all modalities (i.e., voxels, point clouds, multi-view images) and train specific backbones before fusion. However, they still struggle to produce generalized representations due to insufficient 3D training data. Being contrastively pre-trained on web-scale image-text pairs, CLIP inherently produces generalized representations for a wide range of downstream tasks. Building upon it, we present a simple yet effective framework named Describe, Adapt and Combine (DAC) by taking only multi-view images for open-set 3DOR. DAC innovatively synergizes a CLIP model with a multi-modal large language model (MLLM) to learn generalized 3D representations, where the MLLM is used for dual purposes. First, it describes the seen category information to align with CLIP's training objective for adaptation during training. Second, it provides external hints about unknown objects complementary to visual cues during inference. To improve the synergy, we introduce an Additive-Bias Low-Rank adaptation (AB-LoRA), which alleviates overfitting and further enhances the generalization to unseen categories. With only multi-view images, DAC significantly surpasses prior arts by an average of +10.01\% mAP on four open-set 3DOR datasets. Moreover, its generalization is also validated on image-based and cross-dataset setups. Code is available at https://github.com/wangzhichuan123/DAC.

Figures

Figures reproduced from arXiv: 2507.21489 by the authors.

Figure 1
Figure 1. We empower CLIP encoders with an MLLM model [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of DAC. Given a 3D object, we first project it into multi-view images and utilize a pretrained MLLM (e.g., InternVL) to describe the shape information. Next, we efficiently finetune a pretrained CLIP ViT model with a additive-bias LoRA to adapt to the multi-view projections. Finally, we combine the textual and multi-view embeddings to derive 3D descriptors for retrieval. duce discriminative representations … view at source ↗
Figure 3
Figure 3. Comparison of LoRA and Additive-Bias LoRA. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: presents some retrieval examples. Our method can faithfully retrieve related 3D assets for the query. Neverthe￾less, DAC has the following possible limitations. 1) Incor￾porating MLLM at inference increases costs. Yet, note that DAC already outperforms prior art HGM2R …
Figure 5
Figure 5. Figure 5: Impact of fusion weight α on OS-MN40-core. Dataset Backbone α mAP↑ NDCG↑ ANMRR↓ OS-ESB-core ViT-B/32 α = 0.1 58.70 24.27 45.67 ViT-L/14 α = 0.1 57.80 24.36 47.44 OS-NTU-core ViT-B/32 α = 0.6 59.21 27.06 44.58 ViT-L/14 α = 0.3 65.83 28.78 37.46 OS-MN40-core ViT-B/32 α =…
Figure 6
Figure 6. Figure 6: Correlation heatmaps of features from 3 randomly selected classes, each with 10 samples on OS-ABO-core. Darker red indicates [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Retrieval examples on OS-MN40-core. Incorrect matches are in red boxes. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Impact of View Numbers with different backbones. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 46 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  2. [2]

    Learning representations and generative models for 3d point clouds

    Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. pages 40–49, 2018. 1

    work page 2018

  3. [3]

    Flamingo: a visual language model for few-shot learning.NeurIPS, 35: 23716–23736, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.NeurIPS, 35: 23716–23736, 2022. 3

    2022

  4. [4]

    Qwen-vl: A frontier large vision-language model with versatile abilities.arXiv preprint arXiv:2308.12966, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities.arXiv preprint arXiv:2308.12966, 2023. 3

    arXiv 2023

  5. [5]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Jun- yang Lin. Qwen2.5-vl technical repor...

    arXiv 2025

  6. [6]

    Shape matching and object recognition using shape contexts.IEEE TPAMI, 24(4):509–522, 2002

    Serge Belongie, Jitendra Malik, and Jan Puzicha. Shape matching and object recognition using shape contexts.IEEE TPAMI, 24(4):509–522, 2002. 2

    work page 2002

  7. [7]

    On visual similarity based 3d model retrieval

    Ding-Yun Chen, Xiao-Pei Tian, Yu-Te Shen, and Ming Ouhyoung. On visual similarity based 3d model retrieval. InComputer graphics forum, pages 223–232, 2003. 2, 12

    work page 2003

  8. [8]

    Shikra: Unleashing multi- modal llm’s referential dialogue magic.arXiv preprint arXiv:2306.15195, 2023

    Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multi- modal llm’s referential dialogue magic.arXiv preprint arXiv:2306.15195, 2023. 3

    arXiv 2023

Show all 73 references
  1. [9]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. InCVPR, pages 24185–24198, 2024. 1, 2, 3, 4, 6, 7, 13, 17

    2024

  2. [10]

    Pra-net: Point relation-aware network for 3d point cloud analysis.IEEE TIP, 30:4436–4448, 2021

    Silin Cheng, Xiwu Chen, Xinwei He, Zhe Liu, and Xiang Bai. Pra-net: Point relation-aware network for 3d point cloud analysis.IEEE TIP, 30:4436–4448, 2021. 2

    2021

  3. [11]

    Abo: Dataset and benchmarks for real-world 3d object un- derstanding

    Jasmine Collins, Shubham Goel, Kenan Deng, Achlesh- war Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d object un- derstanding. InCVPR, pages 21126–21136, 2022. 12

    2022

  4. [12]

    Siamese cnn-bilstm architecture for 3d shape representation learning

    Guoxian Dai, Jin Xie, Yi Fang, et al. Siamese cnn-bilstm architecture for 3d shape representation learning. InIJCAI, pages 670–676, 2018. 2

    2018

  5. [13]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InCVPR, pages 13142– 13153, 2023. 12, 15

    2023

  6. [14]

    Equivariant multi-view networks

    Carlos Esteves, Yinshuang Xu, Christine Allen-Blanchette, and Kostas Daniilidis. Equivariant multi-view networks. In ICCV, pages 1568–1577, 2019. 2

    2019

  7. [15]

    Gvcnn: Group-view convolutional neural networks for 3d shape recognition

    Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, and Yue Gao. Gvcnn: Group-view convolutional neural networks for 3d shape recognition. InCVPR, pages 264–272, 2018. 1, 2

    2018

  8. [16]

    Meshnet: Mesh neural network for 3d shape rep- resentation

    Yutong Feng, Yifan Feng, Haoxuan You, Xibin Zhao, and Yue Gao. Meshnet: Mesh neural network for 3d shape rep- resentation. InAAAI, pages 8279–8286, 2019. 2

    2019

  9. [17]

    Shrec’22 track: Open-set 3d object retrieval.Computers & Graphics, 107:231–240, 2022

    Yifan Feng, Yue Gao, Xibin Zhao, Yandong Guo, Nihar Bagewadi, Nhat-Tan Bui, Hieu Dao, Shankar Gangisetty, Ripeng Guan, Xie Han, et al. Shrec’22 track: Open-set 3d object retrieval.Computers & Graphics, 107:231–240, 2022. 1, 2, 4

    2022

  10. [18]

    Hypergraph-based multi-modal represen- tation for open-set 3d object retrieval.IEEE TPAMI, 2023

    Yifan Feng, Shuyi Ji, Yu-Shen Liu, Shaoyi Du, Qionghai Dai, and Yue Gao. Hypergraph-based multi-modal represen- tation for open-set 3d object retrieval.IEEE TPAMI, 2023. 1, 2, 4, 5, 6, 8, 12, 15, 17

    2023

  11. [19]

    3d object retrieval based on similarity calculation in 3d computer aided design systems

    Ahmed Fradi, Borhen Louhichi, Mohamed Ali Mahjoub, and Benoit Eynard. 3d object retrieval based on similarity calculation in 3d computer aided design systems. In2017 IEEE/ACS 14th International Conference on Computer Sys- tems and Applications (AICCSA), pages 160–165, 2017. 1

    2017

  12. [20]

    Clip-adapter: Better vision-language models with feature adapters.IJCV, 132(2):581–595, 2024

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters.IJCV, 132(2):581–595, 2024. 2, 3

    2024

  13. [21]

    Deep learning for 3d point clouds: A survey.IEEE TPAMI, 43(12):4338–4364, 2020

    Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey.IEEE TPAMI, 43(12):4338–4364, 2020. 2

    2020

  14. [22]

    3d2seqviews: Aggregating sequential views for 3d global feature learning by cnn with hierarchical attention ag- gregation.IEEE TIP, 28(8):3986–3999, 2019

    Zhizhong Han, Honglei Lu, Zhenbao Liu, Chi-Man V ong, Yu-Shen Liu, Matthias Zwicker, Junwei Han, and CL Philip Chen. 3d2seqviews: Aggregating sequential views for 3d global feature learning by cnn with hierarchical attention ag- gregation.IEEE TIP, 28(8):3986–3999, 2019. 2

    2019

  15. [23]

    Triplet-center loss for multi-view 3d object retrieval

    Xinwei He, Yang Zhou, Zhichao Zhou, Song Bai, and Xiang Bai. Triplet-center loss for multi-view 3d object retrieval. In CVPR, pages 1945–1954, 2018. 6, 15, 17

    1945

  16. [24]

    View n-gram network for 3d object retrieval

    Xinwei He, Tengteng Huang, Song Bai, and Xiang Bai. View n-gram network for 3d object retrieval. InICCV, pages 7515–7524, 2019. 2

    2019

  17. [25]

    Latformer: Locality-aware point- view fusion transformer for 3d shape recognition.PR, 151: 110413, 2024

    Xinwei He, Silin Cheng, Dingkang Liang, Song Bai, Xi Wang, and Yingying Zhu. Latformer: Locality-aware point- view fusion transformer for 3d shape recognition.PR, 151: 110413, 2024. 1

    2024

  18. [26]

    Clip goes 3d: Leveraging prompt tuning for language grounded 3d recognition

    Deepti Hegde, Jeya Maria Jose Valanarasu, and Vishal Pa- tel. Clip goes 3d: Leveraging prompt tuning for language grounded 3d recognition. InICCV, pages 2028–2038, 2023. 3

    2023

  19. [27]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 3, 4, 5

    2022

  20. [28]

    Scal- able deep multimodal learning for cross-modal retrieval

    Peng Hu, Liangli Zhen, Dezhong Peng, and Pei Liu. Scal- able deep multimodal learning for cross-modal retrieval. In Proceedings of the 42nd international ACM SIGIR confer- ence on research and development in information retrieval, pages 635–644, 2019. 6, 15, 17

    2019

  21. [29]

    Clip2point: Transfer clip to point cloud classifica- tion with image-depth pre-training

    Tianyu Huang, Bowen Dong, Yunhan Yang, Xiaoshui Huang, Rynson WH Lau, Wanli Ouyang, and Wangmeng Zuo. Clip2point: Transfer clip to point cloud classifica- tion with image-depth pre-training. InICCV, pages 22157– 22167, 2023. 2, 3

    2023

  22. [30]

    Developing an engineering shape benchmark for cad models.Computer-Aided Design, 38(9):939–953, 2006

    Subramaniam Jayanti, Yagnanarayanan Kalyanaraman, Na- traj Iyer, and Karthik Ramani. Developing an engineering shape benchmark for cad models.Computer-Aided Design, 38(9):939–953, 2006. 7, 12, 13

    2006

  23. [31]

    Cross-modal center loss for 3d cross-modal retrieval

    Longlong Jing, Elahe Vahdani, Jiaxing Tan, and Yingli Tian. Cross-modal center loss for 3d cross-modal retrieval. In CVPR, pages 3142–3151, 2021. 1, 6, 8, 15, 17

    2021

  24. [32]

    Rotationnet: Joint object categorization and pose estimation using multiviews from unsupervised viewpoints

    Asako Kanezaki, Yasuyuki Matsushita, and Yoshifumi Nishida. Rotationnet: Joint object categorization and pose estimation using multiviews from unsupervised viewpoints. InCVPR, pages 5010–5019, 2018. 2

    2018

  25. [33]

    Rotation invariant spherical harmonic repre- sentation of 3 d shape descriptors

    Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz. Rotation invariant spherical harmonic repre- sentation of 3 d shape descriptors. InSymposium on geome- try processing, pages 156–164, 2003. 2

    2003

  26. [34]

    Re- search challenges for digital archives of 3d cultural her- itage models.Journal on Computing and Cultural Heritage (JOCCH), 2(3):1–17, 2010

    David Koller, Bernard Frischer, and Greg Humphreys. Re- search challenges for digital archives of 3d cultural her- itage models.Journal on Computing and Cultural Heritage (JOCCH), 2(3):1–17, 2010. 1

    2010

  27. [35]

    Angular triplet- center loss for multi-view 3d shape retrieval

    Zhaoqun Li, Cheng Xu, and Biao Leng. Angular triplet- center loss for multi-view 3d shape retrieval. InAAAI, pages 8682–8689, 2019. 2

    2019

  28. [36]

    Meshmae: Masked autoencoders for 3d mesh data analysis

    Yaqian Liang, Shanshan Zhao, Baosheng Yu, Jing Zhang, and Fazhi He. Meshmae: Masked autoencoders for 3d mesh data analysis. InECCV, pages 37–54, 2022. 2

    2022

  29. [37]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InCVPR, pages 26296–26306, 2024. 3

    2024

  30. [38]

    Visual instruction tuning.NeurIPS, 36, 2024

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.NeurIPS, 36, 2024. 3

    2024

  31. [39]

    Openshape: Scaling up 3d shape representation towards open-world understanding.NeurIPS, 36, 2023

    Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xu- anlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open-world understanding.NeurIPS, 36, 2023. 3, 6, 12, 15

    2023

  32. [40]

    Point2sequence: Learning the shape representa- tion of 3d point clouds with an attention-based sequence to sequence network

    Xinhai Liu, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. Point2sequence: Learning the shape representa- tion of 3d point clouds with an attention-based sequence to sequence network. InAAAI, pages 8778–8785, 2019. 1

    2019

  33. [41]

    Relation-shape convolutional neural network for point cloud analysis

    Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shape convolutional neural network for point cloud analysis. InCVPR, pages 8895–8904, 2019. 2

    2019

  34. [42]

    An empirical study of scal- ing instruct-tuned large multimodal models.arXiv preprint arXiv:2309.09958, 2023

    Yadong Lu, Chunyuan Li, Haotian Liu, Jianwei Yang, Jian- feng Gao, and Yelong Shen. An empirical study of scal- ing instruct-tuned large multimodal models.arXiv preprint arXiv:2309.09958, 2023. 3

    2023 arXiv

  35. [43]

    3d learning objects for augmented/virtual reality educational ecosystems

    Eleni Mangina. 3d learning objects for augmented/virtual reality educational ecosystems. In2017 23rd International Conference on virtual system & Multimedia (VSMM), pages 1–6, 2017. 1

    2017

  36. [44]

    V oxnet: A 3d con- volutional neural network for real-time object recognition

    Daniel Maturana and Sebastian Scherer. V oxnet: A 3d con- volutional neural network for real-time object recognition. pages 922–928, 2015. 2

    2015

  37. [45]

    Mmjn: Multi-modal joint networks for 3d shape recognition

    Weizhi Nie, Qi Liang, An-An Liu, Zhendong Mao, and Yangyang Li. Mmjn: Multi-modal joint networks for 3d shape recognition. InACM MM, pages 908–916, 2019. 1, 15, 17

    2019

  38. [46]

    Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018. 6, 8, 15, 17

    2018 arXiv

  39. [47]

    Kosmos-2: Ground- ing multimodal large language models to the world.arXiv preprint arXiv:2306.14824, 2023

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Ground- ing multimodal large language models to the world.arXiv preprint arXiv:2306.14824, 2023. 3

    2023 arXiv

  40. [48]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. InCVPR, pages 652–660, 2017. 1, 2

    2017

  41. [49]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space.NeurIPS, 30, 2017

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.NeurIPS, 30, 2017. 2

    2017

  42. [50]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. pages 8748–8763, 2021. 2, 4, 6

    2021

  43. [51]

    Clip for all things zero-shot sketch-based image retrieval, fine- grained or not

    Aneeshan Sain, Ayan Kumar Bhunia, Pinaki Nath Chowd- hury, Subhadeep Koley, Tao Xiang, and Yi-Zhe Song. Clip for all things zero-shot sketch-based image retrieval, fine- grained or not. InCVPR, pages 2765–2775, 2023. 2

    2023

  44. [52]

    Deep- voxels: Learning persistent 3d feature embeddings

    Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, and Michael Zollhofer. Deep- voxels: Learning persistent 3d feature embeddings. In CVPR, pages 2437–2446, 2019. 1

    2019

  45. [53]

    Mv-clip: Multi-view clip for zero-shot 3d shape recog- nition.arXiv preprint arXiv:2311.18402, 2023

    Dan Song, Xinwei Fu, Weizhi Nie, Wenhui Li, and Anan Liu. Mv-clip: Multi-view clip for zero-shot 3d shape recog- nition.arXiv preprint arXiv:2311.18402, 2023. 3, 6, 12

    2023 arXiv

  46. [54]

    Multi-view convolutional neural networks for 3d shape recognition

    Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multi-view convolutional neural networks for 3d shape recognition. InICCV, pages 945–953, 2015. 1, 2

    2015

  47. [55]

    A survey of content based 3d shape retrieval methods.Proceedings Shape Modeling Applications, 2004., pages 145–156, 2004

    Johan WH Tangelder and Remco C Veltkamp. A survey of content based 3d shape retrieval methods.Proceedings Shape Modeling Applications, 2004., pages 145–156, 2004. 2

    2004

  48. [56]

    O-cnn: Octree-based convolutional neural networks for 3d shape analysis.ACM TOG, 36(4):1–11,

    Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, and Xin Tong. O-cnn: Octree-based convolutional neural networks for 3d shape analysis.ACM TOG, 36(4):1–11,

  49. [57]

    The all-seeing project: Towards panop- tic visual recognition and understanding of the open world

    Weiyun Wang, Min Shi, Qingyun Li, Wenhai Wang, Zhen- hang Huang, Linjie Xing, Zhe Chen, Hao Li, Xizhou Zhu, Zhiguo Cao, et al. The all-seeing project: Towards panop- tic visual recognition and understanding of the open world. arXiv preprint arXiv:2308.01907, 2023. 3

    2023 arXiv

  50. [58]

    Visionllm: Large language model is also an open- ended decoder for vision-centric tasks.NeurIPS, 36, 2024

    Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Yu Qiao, et al. Visionllm: Large language model is also an open- ended decoder for vision-centric tasks.NeurIPS, 36, 2024. 3

    2024

  51. [59]

    Dynamic graph cnn for learning on point clouds.ACM TOG, 38(5): 1–12, 2019

    Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds.ACM TOG, 38(5): 1–12, 2019. 2

    2019

  52. [60]

    Teda: Boosting vision-lanuage models for zero-shot 3d object retrieval via testing-time distribution alignment

    Zhichuan Wang, Yang Zhou, Jinhai Xiang, Yulong Wang, and Xinwei He. Teda: Boosting vision-lanuage models for zero-shot 3d object retrieval via testing-time distribution alignment. InProceedings of the 2025 International Confer- ence on Multimedia Retrieval, pages 1442–1451, 2025. 2

    2025

  53. [61]

    View-gcn: View-based graph convolutional network for 3d shape analysis

    Xin Wei, Ruixuan Yu, and Jian Sun. View-gcn: View-based graph convolutional network for 3d shape analysis. InCVPR, pages 1850–1859, 2020. 1

    2020

  54. [62]

    Multi- modal semantic autoencoder for cross-modal retrieval.Neu- rocomputing, 331:165–175, 2019

    Yiling Wu, Shuhui Wang, and Qingming Huang. Multi- modal semantic autoencoder for cross-modal retrieval.Neu- rocomputing, 331:165–175, 2019. 1, 6, 8, 15, 17

    2019

  55. [63]

    3d shapenets: A deep representation for volumetric shapes

    Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Lin- guang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pages 1912–1920, 2015. 1, 2, 12

    1912

  56. [64]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Mart ´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. InCVPR, pages 1179–1189, 2023. 3, 15

    2023

  57. [65]

    Ulip-2: Towards scal- able multimodal pre-training for 3d understanding

    Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Jun- nan Li, Roberto Mart´ın-Mart´ın, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, et al. Ulip-2: Towards scal- able multimodal pre-training for 3d understanding. InCVPR, pages 27091–27101, 2024. 3, 6, 12, 15

    2024

  58. [66]

    Tip-adapter: Training-free clip-adapter for better vision- language modeling.arXiv preprint arXiv:2111.03930, 2021

    Renrui Zhang, Rongyao Fang, Wei Zhang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free clip-adapter for better vision- language modeling.arXiv preprint arXiv:2111.03930, 2021. 3

    2021 arXiv

  59. [67]

    Pointclip: Point cloud understanding by clip

    Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xu- peng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. Pointclip: Point cloud understanding by clip. InCVPR, pages 8552–8562, 2022. 3

    2022

  60. [68]

    Pointweb: Enhancing local neighborhood features for point cloud processing

    Hengshuang Zhao, Li Jiang, Chi-Wing Fu, and Jiaya Jia. Pointweb: Enhancing local neighborhood features for point cloud processing. InCVPR, pages 5565–5573, 2019. 2

    2019

  61. [69]

    Multi-channel weight-sharing autoencoder based on cascade multi-head attention for multimodal emo- tion recognition.IEEE TMM, 2022

    Jiahao Zheng, Sen Zhang, Zilu Wang, Xiaoping Wang, and Zhigang Zeng. Multi-channel weight-sharing autoencoder based on cascade multi-head attention for multimodal emo- tion recognition.IEEE TMM, 2022. 6, 8, 15, 17

    2022

  62. [70]

    Learn- ing placeholders for open-set recognition

    Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Learn- ing placeholders for open-set recognition. InCVPR, pages 4401–4410, 2021. 6, 8, 15, 17

    2021

  63. [71]

    Uni3d: Exploring unified 3d representation at scale

    Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. InICLR, 2024. 6

    2024

  64. [72]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. InCVPR, pages 16816–16825, 2022. 3

    2022

  65. [73]

    a synthetic 3D model view of [cls] with different an- gles

    Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyu Guo, Ziyao Zeng, Zipeng Qin, Shanghang Zhang, and Peng Gao. Point- clip v2: Prompting clip and gpt for powerful 3d open-world learning. InICCV, pages 2639–2650, 2023. 3, 12, 15 Describe, Adapt and Combine: Empowering CLIP Encoders fo...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.