Pith. sign in

REVIEW 3 major objections 5 minor 86 references

Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Frozen image, text, and audio transformers can be adapted to raw 3D point clouds by training only about 3.8% of parameters, outperforming projection-based and 3D-pretrained methods on classification and segmentation benchmarks.

desk verdict A well-engineered PEFT method for 3D with broad experiments, but 'consistently surpasses' and the efficiency claim need an iso-parameter check before they carry weight. read the letter →

arxiv 2509.00374 v1 pith:SRYBYTFE submitted 2025-08-30 cs.CV

Mengke Li , Lihao Chen , Peng Zhang , Yiu-ming Cheung , Hui Huang This is my paper · ORCID
classification cs.CV
keywords pointcloudanalysisparameter-efficientfine-tuningfoundationmodelsprompttuningpermutationinvarianceself-attention3Dclassificationheterogeneousmodalityadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a large pre-trained transformer built for images, text, or audio can be turned into a competitive 3D point cloud classifier without ever flattening the cloud into a 2D image or 1D sequence. The proposed method, APPT, converts raw point groups into tokens with a small learnable embedding network, injects order-invariant position information through a two-parameter layer, and appends a global point-prompt, generated by the same shared-weight module, into every frozen transformer block. Only that module and the task head are trained, about 3.8% of the total parameters. The experiments report accuracy above existing projection-based adapters, 3D-pretrained models, and specialized 3D networks on ScanObjectNN and ModelNet40, including on real noisy scans, with competitive few-shot and segmentation results. If true, this would mean the knowledge embedded in any large 2D or 1D foundation model is directly usable for 3D understanding without projection loss or expensive 3D pre-training.

What carries the argument

The load-bearing mechanism is the weight-sharing pair of a point embedding network and a point-prompt generator, plus the position injector (PosIn). Point_Embed lifts FPS/k-NN point groups into tokens whose dimension matches the frozen transformer; PosIn is a permutation-invariant two-parameter layer that concatenates each token with its deviation from the pooled token centroid and passes the result through a 1D convolution, encoding relative position without learnable dependence on point order; the prompt generator reuses the same weights and applies max-plus-average pooling to emit one global point-prompt per cloud. That prompt is concatenated into every transformer block and carried to th

What would settle it

An iso-parameter experiment would settle it: on ScanObjectNN PB-T50-RS, train APPT with its point embedding scaled so that total trainable parameters equal P2P's 0.25M and Any2Point's 0.8M, keeping PosIn and the shared prompt generator intact. If accuracy falls to those methods' levels, the reported advantage is parameter capacity rather than the prompt-tuning mechanism. Conversely, with the embedding held fixed at APPT's 3.4M budget, the ablation must reproduce the roughly one-point-per-module drops shown in Table 4 when PosIn or the prompt generator is removed; if removing them changes nothi

Watch

Extended reading notes

Core claim

The paper claims that heterogeneous foundation models can be fine-tuned for point cloud analysis directly in the point domain, with no lossy mapping to 2D or 1D. APPT groups raw points by farthest-point sampling and k-nearest neighbors, embeds each group with a Point-PN-style network, and injects relative position through the position injector, whose only parameters are a 1D convolution over the concatenation of a token and its deviation from the pooled token centroid. A prompt generator sharing the embedding network's weights pools the tokens into one global point-prompt, concatenated into every transformer block and propagated forward, so the frozen backbone's self-attention is recalibrate

Load-bearing premise

The load-bearing premise is that the reported gains come from the position injector and the shared prompt generator rather than from APPT's larger trainable-parameter budget: the paper compares against P2P (0.25M) and Any2Point (0.8M) while APPT trains 3.4M, with no iso-parameter ablation isolating the modules (Sec. 4.3, Table 7). If embedding capacity, not the tuning mechanism, drives the gap, the central efficiency-and-superiority claim weakens.

Editorial extensions

If this is right

  • Any frozen transformer, whether trained on images, text, or audio, can be repurposed for 3D point cloud classification by training a small shared point-embedding module, eliminating lossy 3D-to-2D or 3D-to-1D projection.
  • On the harder real-scan benchmark (ScanObjectNN PB-T50-RS), the paper reports 92.6% accuracy with an image prior, roughly 5.5 points above the Point-PN backbone and above projection-based competitors such as P2P and Any2Point.
  • Few-shot classification with the frozen backbone stays above roughly 92% accuracy at 10 shots per class, suggesting the heterogeneous priors transfer reliably when labels are scarce.
  • Part segmentation on ShapeNetPart reaches competitive mean IoU with a fraction of the trainable parameters of 3D-pretrained models such as Point-MAE and Joint-MAE, although not the top published accuracy.
  • Because the method works across ViT, DINOv2, DeiT, CLIP text, RoBERTa, and ImageBind audio backbones without architectural change, the adaptation is modality-agnostic by construction.
  • The shared-weight prompt generator means the added trainable parameter cost over a specialized 3D network stays small even as the frozen backbone grows, so the approach scales to larger foundation models without proportional tuning cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline efficiency comparison mixes parameter budgets: APPT trains 3.4M parameters, P2P trains 0.25M, and Any2Point trains 0.8M, so an iso-parameter ablation is the natural next experiment to decide whether the gains come from the prompt-tuning modules or from the larger Point-PN embedding they sit on.
  • The permutation-invariance proof covers the pooling operations and the global prompt, but not the per-point token sequence fed to the transformer; an inference-time shuffle test would tell whether order invariance holds end-to-end.
  • The shared-weight design suggests a route the paper does not explore: pre-train the point-embedding network once on large point data, then attach it to any frozen 2D or 1D transformer, making multi-modal 3D analysis a plug-and-play operation.
  • Because the point-prompt is a single global feature per cloud, heavily occluded or partial scans that distort global geometry would likely stress the method more than object classification does; partial-view or real-world robustness benchmarks would quantify that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Adaptive Point-Prompt Tuning (APPT), a parameter-efficient fine-tuning method that adapts frozen pre-trained transformers from heterogeneous modalities (image, text, audio) to direct 3D point cloud classification and segmentation. APPT consists of a Point-PN-based point embedding module, a permutation-invariant position injector (PosIn), and a weight-shared prompt generator that injects a global point-prompt into every transformer block. The authors claim that APPT consistently surpasses existing methods while fine-tuning only 3.8% of parameters, and they support this with experiments on ScanObjectNN, ModelNet40, ShapeNetPart, few-shot classification, and ablations across multiple backbone models. A conference version of the work was published as APF [27].

Significance. If the central claims hold, APPT would be a practical contribution: it enables frozen 1D/2D foundation models to process raw 3D point clouds without lossy projection, using a small learnable module and a shared prompt generator. The paper's strengths are the breadth of the empirical evaluation (multiple modalities, multiple backbones, three downstream tasks), the explicit permutation-invariance proofs in Sec. 3.5, and the public code release. However, the efficiency and 'consistently surpasses' claims are currently overstated relative to the evidence: the main accuracy gains are obtained with substantially more trainable parameters than the strongest parameter-efficient baselines, and several reported numbers contradict the superiority claim. The paper is a credible incremental advance over APF, but the central comparative claims need to be re-scoped or additionally supported before acceptance.

major comments (3)
  1. [Sec. 4.3, Tables 4 and 7] The central efficiency/superiority claim is not established because the comparison does not hold the trainable-parameter budget fixed. Table 7 lists APPT at 3.4M trainable parameters versus 0.25M for P2P and 0.8M for Any2Point. The extra capacity comes almost entirely from the larger Point-PN-based point embedding module, which is also used, with fewer trainable parameters, by Any2Point. The ablation in Table 4 adds PosIn and point-prompt tuning incrementally, but never keeps parameter count constant; hence it cannot separate the effect of the proposed modules from the effect of increased model capacity. Please add an iso-parameter ablation (e.g., a P2P or Any2Point variant with a comparably sized embedding, or an APPT variant with a smaller Point-PN embedding matched to 0.8M params) to show that the accuracy improvement is attributable to the APPT mechanism rather than to the larger tra
  2. [Sec. 4.2, Tables 3 and 5; Abstract] The claim that APPT consistently surpasses existing methods is contradicted by the paper's own results. In Table 5, with the CLIP text prior on ScanObjectNN PB-T50-RS, APPT scores 91.4% versus Any2Point's 91.9%; with RoBERTa, APPT scores 87.3% versus Any2Point's 89.7%. In Table 3, APPT trails Joint-MAE on both part-segmentation metrics (84.0 vs. 85.4 mIoU_C; 85.9 vs. 86.3 mIoU_I). The text acknowledges that APPT slightly lags or may not outperform in these cases, but the abstract and introduction state consistent superiority. Please rephrase the central claim to be accurate, for example by stating APPT is competitive or superior in most settings with notable gains on ScanObjectNN with 2D priors, and explicitly discuss the settings where it is behind.
  3. [Sec. 4.2, Tables 1 and 2] Reported gains are small or absent in several settings, which further undermines the consistent superiority claim. On ModelNet40, APPT with the 2D ViT-B prior reaches 94.2%, equal to the conference version APF and only +0.4% over the Point-PN base; in the 10-way 20-shot few-shot setting, APPT (2D) at 95.3% underperforms APF at 95.7%. These results are not necessarily disqualifying, but they should be presented with appropriate hedging and, where possible, error bars for the full classification settings in Table 1 to establish that the differences are meaningful. The few-shot table already reports standard deviations, so the absence of variability estimates in Table 1 is a notable gap.
minor comments (5)
  1. [Sec. 4.2, Table 2] The reference label for APF is inconsistent: Table 2 cites APF [46], while the conference version is [27] and Table 1 cites APF [27]. Please correct the citation.
  2. [Sec. 4.2, Table 3] The table header for per-category IoU is difficult to read because columns are not aligned with the category names. Consider formatting the table with two-line headers or separating the per-category columns for clarity.
  3. [Sec. 3.4, Eq. (12)] The notation Pool(.) is defined as the sum of max and average pooling, but the same operation is also used in Eq. (8) for the prompt generator. It may help to introduce a named function (e.g., MixPool) to avoid ambiguity.
  4. [General] There are minor typographical issues, such as 'the's generalization' in Sec. 2.2 and 'L-st' in Fig. 2. A copyediting pass is recommended.
  5. [Sec. 3.5] The proof of Theorem 3 is correct, but the statement that maxP and avgP are both permutation-invariant functions should explicitly note that the invariance is over the token (row) dimension, not the channel dimension. This is implied by the notation, but an explicit statement would prevent misinterpretation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the method's gains are measured on external benchmarks; self-citations are disclosed prior work and do not force any result.

full rationale

The paper makes empirical claims evaluated on held-out test sets (ScanObjectNN, ModelNet40, ShapeNetPart) against external baselines, so the central numbers do not reduce to the method's own equations. The method is defined constructively (point embedding Eq. 4, PosIn Eqs. 5-7, prompt generation Eq. 8, block recursion Eqs. 9-12); performance is not an algebraic consequence of these definitions. The only formal derivation, Theorem 3 in Sec. 3.5, proves permutation invariance of max/mean pooling, which is a standard property and does not assume the theorem's conclusion. Self-citations are present ([27] APF conference version; [69] same-authors visual prompt tuning) but they are disclosed prior work, used as a comparison baseline or as design inspiration, not as a load-bearing theorem; no uniqueness claim or 'forbidden alternative' is imported from them. The paper itself flags limitations: in Sec. 4.2 it states 'APPT may not outperform SOTA methods across all evaluation metrics' (Table 3), and in Sec. 4.3 it concedes 'APPT slightly lags behind Any2Point when using textual pre-trained knowledge on ScanObjectNN (91.4% vs. 91.7% and 87.3% vs. 89.7%)' (Table 5). These passages, together with the unequal parameter counts in Table 7 (APPT 3.4M vs Any2Point 0.8M vs P2P 0.25M), undermine the breadth of the 'consistently surpasses' and efficiency claims, but they are correctness/fairness concerns, not circularity: the compared accuracies are externally measured. No fitted parameter is relabeled as a prediction, no result is defined in terms of its target, and no step reduces by construction to its inputs. Therefore score 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

APPT is an empirical ML method; its central claim rests on the usual assumption that frozen transformer backbones trained on images, text, or audio carry transferable priors, and on the correctness of standard set-pooling facts used in the permutation-invariance argument. No hidden fitted constants are introduced beyond learned weights and standard hyperparameters. The main unspecified choices are the FPS sample count Ns and kNN size k, which are not reported in the text.

free parameters (5)
  • Number of sampled centroid points Ns
    FPS sampling count not specified in the text; affects token count and accuracy.
  • Number of neighbors k for kNN grouping
    Not specified; standard values likely follow Point-PN.
  • Point embedding output dimension = 768
    Set to match ViT-B embedding dimension; a design hyperparameter.
  • Number of prompt tokens = 1
    The paper uses a single point-prompt per block; prompt count is a design choice.
  • Position injector coefficients a and b
    Learnable, implemented as a 1D convolution; trained on the target dataset.
assumptions (3)
  • domain assumption Frozen pre-trained transformer backbones (ViT, CLIP, RoBERTa, ImageBind) carry transferable priors useful for 3D point cloud tasks.
    This is the central premise of PEFT-for-3D; tested through experiments but not proven.
  • domain assumption FPS and kNN grouping produce a stable and representative token set for arbitrary input point clouds.
    Standard geometric grouping assumed throughout Sec. 3.1-3.2.
  • standard math Max and mean functions are permutation-invariant (Lemmas 1-2, Sec. 3.5).
    Elementary facts used to prove Theorem 3 about pooled features.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis." pith.science (2026). https://pith.science/paper/SRYBYTFE

@misc{pith2026250900374,
  author       = {Pith},
  title        = {Pith review of: Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SRYBYTFE}},
  note         = {Machine review of arXiv:2509.00374}
}
read the original abstract

Parameter-efficient fine-tuning strategies for foundation models in 1D textual and 2D visual analysis have demonstrated remarkable efficacy. However, due to the scarcity of point cloud data, pre-training large 3D models remains a challenging task. While many efforts have been made to apply pre-trained visual models to 3D domains through "high-to-low" mapping, these approaches often lead to the loss of spatial geometries and lack a generalizable framework for adapting any modality to 3D. This paper, therefore, attempts to directly leverage point features to calibrate the heterogeneous foundation model of any modality for 3D point cloud analysis. Specifically, we propose the Adaptive Point-Prompt Tuning (APPT) method, which fine-tunes pre-trained models with a modest number of parameters, enabling direct point cloud processing without heterogeneous mappings. We convert raw point clouds into point embeddings by aggregating local geometry to capture spatial features followed by linear layers to ensure seamless utilization of frozen pre-trained models. Given the inherent disorder of point clouds, in contrast to the structured nature of images and language, we employ a permutation-invariant feature to capture the relative positions of point embeddings, thereby obtaining point tokens enriched with location information to optimize self-attention mechanisms. To calibrate self-attention across source domains of any modality to 3D and reduce computational overhead, we introduce a prompt generator that shares weights with the point embedding module, dynamically producing point-prompts without adding additional parameters. These prompts are then concatenated into a frozen foundation model, providing rich global structural information and compensating for the lack of structural context in the heterogeneous data.

Figures

Figures reproduced from arXiv: 2509.00374 by the authors.

Figure 1
Figure 1. Comparison between existing methods and our pro [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The structure of our proposed adaptive point-prompt tuning. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The structure of our proposed position injector [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: T-SNE visualization of feature distributions. We show [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Visualization of the effectiveness of different mod [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

86 extracted references · 76 canonical work pages

  1. [27]

    Adapt point- former: 3d point cloud analysis via adapting 2d visual transform- ers,

    M. Li, D. Li, G. Yang, Y. Cheung, and H. Huang, “Adapt point- former: 3d point cloud analysis via adapting 2d visual transform- ers,” in Eur. Conf. Artif. Intell., vol. 392, 2024, pp. 89–96

    work page 2024

  2. [1]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. E. Hinton, “A simple framework for contrastive learning of visual representations,” in Int. Conf. Mach. Learn., 2020, pp. 1597–1607

    2020

  3. [2]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y. Shen, P . Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in Int. Conf. Learn. Represent., 2022

    2022

  4. [3]

    Adaptformer: Adapting vision transformers for scalable visual recognition,

    S. Chen, C. Ge, Z. Tong, J. Wang, Y. Song, J. Wang, and P . Luo, “Adaptformer: Adapting vision transformers for scalable visual recognition,” Adv. Neural Inform. Process. Syst., vol. 35, pp. 16 664– 16 678, 2022

    2022

  5. [4]

    Visual tuning,

    B. X. Yu, J. Chang, H. Wang, L. Liu, S. Wang, Z. Wang, J. Lin, L. Xie, H. Li, Z. Lin et al., “Visual tuning,” ACM Comput. Surv. , vol. 56, no. 12, pp. 297:1–297:38, 2024

    2024

  6. [5]

    Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,

    P . Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” ACM Comput. Surv. , vol. 55, no. 9, pp. 1–35, 2023

    2023

  7. [6]

    Bert: Pre- training of deep bidirectional transformers for language under- standing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre- training of deep bidirectional transformers for language under- standing,” arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

  8. [7]

    Gpt-3: Its nature, scope, limits, and consequences,

    L. Floridi and M. Chiriatti, “Gpt-3: Its nature, scope, limits, and consequences,” Minds and Machines, vol. 30, pp. 681–694, 2020

    2020

Show all 86 references
  1. [8]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    D. Alexey, B. Lucas, K. Alexander, W. Dirk, Z. Xiaohua, U. Thomas, D. Mostafa, M. Matthias, H. Georg, G. Sylvain et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Int. Conf. Learn. Represent., 2021

    2021

  2. [9]

    Learning trans- ferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P . Mishkin, J. Clark et al. , “Learning trans- ferable visual models from natural language supervision,” in Int. Conf. Learn. Represent., 2021, pp. 8748–8763

    2021

  3. [10]

    DINOv2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V . Khalidov, P . Fernandez, D. Haziza, F. Massa, A. El-Noubyet al., “DINOv2: Learning robust visual features without supervision,” Trans. Mach. Learn. Res., vol. 2024, 2024

    2024

  4. [11]

    Deep learning for 3d point clouds: A survey,

    Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu, and M. Bennamoun, “Deep learning for 3d point clouds: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 12, pp. 4338–4364, 2020

    2020

  5. [12]

    Point- bert: Pre-training 3d point cloud transformers with masked point modeling,

    X. Yu, L. Tang, Y. Rao, T. Huang, J. Zhou, and J. Lu, “Point- bert: Pre-training 3d point cloud transformers with masked point modeling,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2022, pp. 19 313–19 322

    2022

  6. [13]

    Unsuper- vised point cloud pre-training via occlusion completion,

    H. Wang, Q. Liu, X. Yue, J. Lasenby, and M. J. Kusner, “Unsuper- vised point cloud pre-training via occlusion completion,” in Int. Conf. Comput. Vis., 2021, pp. 9782–9792

    2021

  7. [14]

    Pointgpt: Auto-regressively generative pre-training from point clouds,

    G. Chen, M. Wang, Y. Yang, K. Yu, L. Yuan, and Y. Yue, “Pointgpt: Auto-regressively generative pre-training from point clouds,” in Adv. Neural Inform. Process. Syst., vol. 36, 2023, pp. 29 667–29 679

    2023

  8. [15]

    Any2point: Empowering any-modality large models for efficient 3d understanding,

    Y. Tang, R. Zhang, J. Liu, Z. Guo, B. Zhao, Z. Wang, P . Gao, H. Li, D. Wang, and X. Li, “Any2point: Empowering any-modality large models for efficient 3d understanding,” in Eur. Conf. Comput. Vis., 2024, pp. 456–473

    2024

  9. [16]

    P2P: tuning pre- trained image models for point cloud analysis with point-to-pixel prompting,

    Z. Wang, X. Yu, Y. Rao, J. Zhou, and J. Lu, “P2P: tuning pre- trained image models for point cloud analysis with point-to-pixel prompting,” Adv. Neural Inform. Process. Syst., 2022

    2022

  10. [17]

    Flattening-net: Deep regular 2d representation for 3d point cloud analysis,

    Q. Zhang, J. Hou, Y. Qian, Y. Zeng, J. Zhang, and Y. He, “Flattening-net: Deep regular 2d representation for 3d point cloud analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 8, pp. 9726–9742, 2023

    2023

  11. [18]

    Point-to-pixel prompt- ing for point cloud analysis with pre-trained image models,

    Z. Wang, Y. Rao, X. Yu, J. Zhou, and J. Lu, “Point-to-pixel prompt- ing for point cloud analysis with pre-trained image models,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 6, pp. 4381–4397, 2024

    2024

  12. [19]

    Pointllm: Empowering large language models to understand point clouds,

    R. Xu, X. Wang, T. Wang, Y. Chen, J. Pang, and D. Lin, “Pointllm: Empowering large language models to understand point clouds,” in Eur. Conf. Comput. Vis., 2024, pp. 131–147

    2024

  13. [20]

    Learning 3D representations from 2D pre-trained models via image-to-point masked autoencoders,

    R. Zhang, L. Wang, Y. Qiao, P . Gao, and H. Li, “Learning 3D representations from 2D pre-trained models via image-to-point masked autoencoders,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2023, pp. 21 769–21 780

    2023

  14. [21]

    Openshape: Scaling up 3d shape representation to- wards open-world understanding,

    M. Liu, R. Shi, K. Kuang, Y. Zhu, X. Li, S. Han, H. Cai, F. Porikli, and H. Su, “Openshape: Scaling up 3d shape representation to- wards open-world understanding,” in Adv. Neural Inform. Process. Syst., vol. 36, 2023, pp. 44 860–44 879

    2023

  15. [22]

    Partdistill: 3d shape part segmentation by vision-language model distillation,

    A. Umam, C.-K. Yang, M.-H. Chen, J.-H. Chuang, and Y.-Y. Lin, “Partdistill: 3d shape part segmentation by vision-language model distillation,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog. , 2024, pp. 3470–3479

    2024

  16. [23]

    ULIP: Learning a unified representation of language, images, and point clouds for 3d un- derstanding,

    L. Xue, M. Gao, C. Xing, R. Martín-Martín, J. Wu, C. Xiong, R. Xu, J. C. Niebles, and S. Savarese, “ULIP: Learning a unified representation of language, images, and point clouds for 3d un- derstanding,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog. , 2023, pp. 1179–1189

    2023

  17. [24]

    ULIP-2: Towards scalable multimodal pre-training for 3d understanding,

    L. Xue, N. Yu, S. Zhang, A. Panagopoulou, J. Li, R. Martín-Martín, J. Wu, C. Xiong, R. Xu, J. C. Niebles, and S. Savarese, “ULIP-2: Towards scalable multimodal pre-training for 3d understanding,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog. , June 2024, pp. 27 091–27 101

    2024

  18. [25]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space,

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” Adv. Neural Inform. Process. Syst., vol. 30, 2017

    2017

  19. [26]

    Deep sets,

    M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhut- dinov, and A. J. Smola, “Deep sets,” Adv. Neural Inform. Process. Syst., vol. 30, 2017

    2017

  20. [28]

    Pointnet: Deep learning on point sets for 3d classification and segmentation,

    C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2017, pp. 652–660

    2017

  21. [29]

    Point-voxel cnn for efficient 3d deep learning,

    Z. Liu, H. Tang, Y. Lin, and S. Han, “Point-voxel cnn for efficient 3d deep learning,” Adv. Neural Inform. Process. Syst., vol. 32, 2019

    2019

  22. [30]

    Pv- rcnn: Point-voxel feature set abstraction for 3d object detection,

    S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “Pv- rcnn: Point-voxel feature set abstraction for 3d object detection,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2020, pp. 10 529–10 538

    2020

  23. [31]

    Surface representation for point clouds,

    H. Ran, J. Liu, and C. Wang, “Surface representation for point clouds,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog. , 2022, pp. 18 942–18 952

    2022

  24. [32]

    Bevdepth: Acquisition of reliable depth for multi-view 3d object detection,

    Y. Li, Z. Ge, G. Yu, J. Yang, Z. Wang, Y. Shi, J. Sun, and Z. Li, “Bevdepth: Acquisition of reliable depth for multi-view 3d object detection,” in AAAI Conf. Artif. Intell., vol. 37, no. 2, 2023, pp. 1477– 1485

    2023

  25. [33]

    Pointnext: Revisiting pointnet++ with improved training and scaling strategies,

    G. Qian, Y. Li, H. Peng, J. Mai, H. Hammoud, M. Elhoseiny, and B. Ghanem, “Pointnext: Revisiting pointnet++ with improved training and scaling strategies,” Adv. Neural Inform. Process. Syst. , vol. 35, pp. 23 192–23 204, 2022

    2022

  26. [34]

    Dynamic graph cnn for learning on point clouds,

    Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” ACM Trans. Graph., vol. 38, no. 5, pp. 1–12, 2019

    2019

  27. [35]

    Kpconv: Flexible and deformable convolution for point clouds,

    H. Thomas, C. R. Qi, J.-E. Deschaud, B. Marcotegui, F. Goulette, and L. J. Guibas, “Kpconv: Flexible and deformable convolution for point clouds,” in Int. Conf. Comput. Vis., 2019, pp. 6411–6420

    2019

  28. [36]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Adv. Neural Inform. Process. Syst., vol. 30, 2017

    2017

  29. [37]

    Point trans- former,

    H. Zhao, L. Jiang, J. Jia, P . H. Torr, and V . Koltun, “Point trans- former,” in Int. Conf. Comput. Vis., 2021, pp. 16 259–16 268

    2021

  30. [38]

    Pct: Point cloud transformer,

    M.-H. Guo, J.-X. Cai, Z.-N. Liu, T.-J. Mu, R. R. Martin, and S.-M. Hu, “Pct: Point cloud transformer,” Comput. Vis. Media, vol. 7, pp. 187–199, 2021

    2021

  31. [39]

    Pointmixer: Mlp-mixer for point cloud understanding,

    J. Choe, C. Park, F. Rameau, J. Park, and I. S. Kweon, “Pointmixer: Mlp-mixer for point cloud understanding,” in Eur. Conf. Comput. Vis., 2022, pp. 620–640

    2022

  32. [40]

    Point transformer v2: Grouped vector attention and partition-based pooling,

    X. Wu, Y. Lao, L. Jiang, X. Liu, and H. Zhao, “Point transformer v2: Grouped vector attention and partition-based pooling,” Adv. Neural Inform. Process. Syst., vol. 35, pp. 33 330–33 342, 2022

    2022

  33. [41]

    Condaformer: Disassembled transformer with local structure en- hancement for 3d point cloud understanding,

    L. Duan, S. Zhao, N. Xue, M. Gong, G.-S. Xia, and D. Tao, “Condaformer: Disassembled transformer with local structure en- hancement for 3d point cloud understanding,” Adv. Neural Inform. Process. Syst., vol. 36, 2023. SUBMITTED TO IEEE TRANSACTIONS ON PATTERN ANAL YSIS AND MAC...

    2023

  34. [42]

    Mamba3d: Enhancing local features for 3d point cloud analysis via state space model,

    X. Han, Y. Tang, Z. Wang, and X. Li, “Mamba3d: Enhancing local features for 3d point cloud analysis via state space model,” in ACM Int. Conf. Multimedia, 2024, pp. 4995–5004

    2024

  35. [43]

    Point transformer v3: Simpler faster stronger,

    X. Wu, L. Jiang, P .-S. Wang, Z. Liu, X. Liu, Y. Qiao, W. Ouyang, T. He, and H. Zhao, “Point transformer v3: Simpler faster stronger,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4840–4851

    2024

  36. [44]

    Multimodal token fusion for vision transformers,

    Y. Wang, X. Chen, L. Cao, W. Huang, F. Sun, and Y. Wang, “Multimodal token fusion for vision transformers,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2022, pp. 12 186–12 195

    2022

  37. [45]

    Pointofview: A multi-modal network for few-shot 3d point cloud classification fusing point and multi-view image features,

    H. Ren, J. Wang, M. Yang, and S. Velipasalar, “Pointofview: A multi-modal network for few-shot 3d point cloud classification fusing point and multi-view image features,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2024, pp. 784–793

    2024

  38. [46]

    Ashapeformer: Semantics-guided object-level active shape encoding for 3d object detection via transformers,

    Z. Li, H. Yu, Z. Yang, T. Chen, and N. Akhtar, “Ashapeformer: Semantics-guided object-level active shape encoding for 3d object detection via transformers,” inIEEE/CVF Conf. Comput. Vis. Pattern Recog., 2023, pp. 1012–1021

    2023

  39. [47]

    Point cloud pre-training with diffusion models,

    X. Zheng, X. Huang, G. Mei, Y. Hou, Z. Lyu, B. Dai, W. Ouyang, and Y. Gong, “Point cloud pre-training with diffusion models,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2024, pp. 22 935–22 945

    2024

  40. [48]

    Point-peft: Parameter-efficient fine-tuning for 3d pre- trained models,

    Y. Tang, R. Zhang, Z. Guo, X. Ma, B. Zhao, Z. Wang, D. Wang, and X. Li, “Point-peft: Parameter-efficient fine-tuning for 3d pre- trained models,” in AAAI Conf. Artif. Intell. , vol. 38, no. 6, 2024, pp. 5171–5179

    2024

  41. [49]

    Masked autoencoders for point cloud self-supervised learning,

    Y. Pang, W. Wang, F. E. Tay, W. Liu, Y. Tian, and L. Yuan, “Masked autoencoders for point cloud self-supervised learning,” in Eur. Conf. Comput. Vis., 2022, pp. 604–621

    2022

  42. [50]

    Point-m2ae: Multi-scale masked autoencoders for hi- erarchical point cloud pre-training,

    R. Zhang, Z. Guo, P . Gao, R. Fang, B. Zhao, D. Wang, Y. Qiao, and H. Li, “Point-m2ae: Multi-scale masked autoencoders for hi- erarchical point cloud pre-training,” in Adv. Neural Inform. Process. Syst., vol. 35, 2022, pp. 27 061–27 074

    2022

  43. [51]

    PointCLIP: Point cloud understanding by clip,

    R. Zhang, Z. Guo, W. Zhang, K. Li, X. Miao, B. Cui, Y. Qiao, P . Gao, and H. Li, “PointCLIP: Point cloud understanding by clip,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2022, pp. 8542–8552

    2022

  44. [52]

    Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning,

    X. Zhu, R. Zhang, B. He, Z. Guo, Z. Zeng, Z. Qin, S. Zhang, and P . Gao, “Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning,” in Int. Conf. Comput. Vis. , 2023, pp. 2639– 2650

    2023

  45. [53]

    View-GCN: View-based graph convolu- tional network for 3D shape analysis,

    X. Wei, R. Yu, and J. Sun, “View-GCN: View-based graph convolu- tional network for 3D shape analysis,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2020, pp. 1847–1856

    2020

  46. [54]

    Predicting the perceptual quality of point cloud: A 3d-to-2d projection-based exploration,

    Q. Yang, H. Chen, Z. Ma, Y. Xu, R. Tang, and J. Sun, “Predicting the perceptual quality of point cloud: A 3d-to-2d projection-based exploration,” IEEE Trans. Multimedia, vol. 23, pp. 3877–3891, 2020

    2020

  47. [55]

    Data efficient 3d learner via knowledge transferred from 2d model,

    P .-C. Yu, C. Sun, and M. Sun, “Data efficient 3d learner via knowledge transferred from 2d model,” in Eur. Conf. Comput. Vis., 2022, pp. 182–198

    2022

  48. [56]

    Autoencoders as cross-modal teachers: Can pretrained 2d image transformers help 3d representation learning?

    R. Dong, Z. Qi, L. Zhang, J. Zhang, J. Sun, Z. Ge, L. Yi, and K. Ma, “Autoencoders as cross-modal teachers: Can pretrained 2d image transformers help 3d representation learning?” in Int. Conf. Learn. Represent., 2023

    2023

  49. [57]

    Visual prompt tuning,

    M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim, “Visual prompt tuning,” in Eur. Conf. Comput. Vis., 2022, pp. 709–727

    2022

  50. [58]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, and N. H. Jakob Uszkoreit, “An image is worth 16x16 words: Transformers for image recognition at scale,” in Int. Conf. Learn. Represent., 2021

    2021

  51. [59]

    Developing real-time streaming transformer transducer for speech recognition on large- scale dataset,

    X. Chen, Y. Wu, Z. Wang, S. Liu, and J. Li, “Developing real-time streaming transformer transducer for speech recognition on large- scale dataset,” in IEEE Int. Conf. Acoust. Speech Signal Process., 2021, pp. 5904–5908

    2021

  52. [60]

    Layer normalization,

    J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016

    2016 arXiv

  53. [61]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2016, pp. 770–778

    2016

  54. [62]

    Rethinking network design and local geometry in point cloud: A simple residual mlp framework,

    X. Ma, C. Qin, H. You, H. Ran, and Y. Fu, “Rethinking network design and local geometry in point cloud: A simple residual mlp framework,” in Int. Conf. Learn. Represent., 2022

    2022

  55. [63]

    Starting from non-parametric networks for 3d point cloud analysis,

    R. Zhang, L. Wang, Y. Wang, P . Gao, H. Li, and J. Shi, “Starting from non-parametric networks for 3d point cloud analysis,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2023, pp. 5344–5353

    2023

  56. [64]

    Tokens-to-token vit: Training vision transformers from scratch on imagenet,

    L. Yuan, Y. Chen, T. Wang, W. Yu, Y. Shi, Z.-H. Jiang, F. E. Tay, J. Feng, and S. Yan, “Tokens-to-token vit: Training vision transformers from scratch on imagenet,” in Int. Conf. Comput. Vis., October 2021, pp. 558–567

    2021

  57. [65]

    Explor- ing visual prompts for adapting large-scale models,

    H. Bahng, A. Jahanian, S. Sankaranarayanan, and P . Isola, “Explor- ing visual prompts for adapting large-scale models,”arXiv preprint arXiv:2203.17274, 2022

    2022 arXiv

  58. [66]

    Read-only prompt optimization for vision-language few-shot learning,

    D. Lee, S. Song, J. Suh, J. Choi, S. Lee, and H. J. Kim, “Read-only prompt optimization for vision-language few-shot learning,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2023, pp. 1401–1411

    2023

  59. [67]

    LPT: Long-tailed prompt tuning for image classification,

    B. Dong, P . Zhou, S. Yan, and W. Zuo, “LPT: Long-tailed prompt tuning for image classification,” in Int. Conf. Learn. Represent., 2022

    2022

  60. [68]

    Long- tail learning with foundation model: Heavy fine-tuning hurts,

    J.-X. Shi, T. Wei, Z. Zhou, J.-J. Shao, X.-Y. Han, and Y.-F. Li, “Long- tail learning with foundation model: Heavy fine-tuning hurts,” in Forty-first International Conference on Machine Learning , 2024

    2024

  61. [69]

    Improving visual prompt tuning by gaussian neighborhood minimization for long-tailed visual recognition,

    M. Li, Y. Liu, Y. Lu, Y. Zhang, Y.-m. Cheung, and H. Huang, “Improving visual prompt tuning by gaussian neighborhood minimization for long-tailed visual recognition,” arXiv preprint arXiv:2410.21042, 2024

    2024 arXiv

  62. [70]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P . Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in ACL/IJCNLP, 2021, pp. 4582–4597

    2021

  63. [71]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Int. Conf. Comput. Vis., 2021, pp. 10 012–10 022

    2021

  64. [72]

    Deep sets,

    M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdi- nov, and A. J. Smola, “Deep sets,” in Adv. Neural Inform. Process. Syst., vol. 30, 2017

    2017

  65. [73]

    Deep set prediction networks,

    Y. Zhang, J. Hare, and A. Prugel-Bennett, “Deep set prediction networks,” in Adv. Neural Inform. Process. Syst., vol. 32, 2019

    2019

  66. [74]

    Joint-mae: 2d-3d joint masked autoencoders for 3d point cloud pre-training,

    G. Ziyu, Z. Renrui, Q. Longtian, L. Xianzhi, and H. Pheng-Ann, “Joint-mae: 2d-3d joint masked autoencoders for 3d point cloud pre-training,” in Int. Joint Conf. Artif. Intell., 2023, pp. 791–799

    2023

  67. [75]

    Dy- namic adapter meets prompt tuning: Parameter-efficient transfer learning for point cloud analysis,

    X. Zhou, D. Liang, W. Xu, X. Zhu, Y. Xu, Z. Zou, and X. Bai, “Dy- namic adapter meets prompt tuning: Parameter-efficient transfer learning for point cloud analysis,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2024, pp. 14 707–14 717

    2024

  68. [76]

    Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining,

    Z. Qi, R. Dong, G. Fan, Z. Ge, X. Zhang, K. Ma, and L. Yi, “Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining,” in Int. Conf. Mach. Learn. , 2023, pp. 28 223–28 243

    2023

  69. [77]

    Crosspoint: Self-supervised cross- modal contrastive learning for 3d point cloud understanding,

    M. Afham, I. Dissanayake, D. Dissanayake, A. Dharmasiri, K. Thi- lakarathna, and R. Rodrigo, “Crosspoint: Self-supervised cross- modal contrastive learning for 3d point cloud understanding,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog. , June 2022, pp. 9902– 9912

    2022

  70. [78]

    Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data,

    M. A. Uy, Q.-H. Pham, B.-S. Hua, T. Nguyen, and S.-K. Yeung, “Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data,” in Int. Conf. Comput. Vis., 2019, pp. 1588–1597

    2019

  71. [79]

    3D shapenets: A deep representation for volumetric shapes,

    Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, “3D shapenets: A deep representation for volumetric shapes,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2015, pp. 1912–1920

    2015

  72. [80]

    A scalable active framework for region annotation in 3d shape collections,

    L. Yi, V . G. Kim, D. Ceylan, I.-C. Shen, M. Yan, H. Su, C. Lu, Q. Huang, A. Sheffer, and L. Guibas, “A scalable active framework for region annotation in 3d shape collections,” ACM Trans. Graph., vol. 35, no. 6, pp. 1–12, 2016

    2016

  73. [81]

    Paconv: Position adaptive convolution with dynamic kernel assembling on point clouds,

    M. Xu, R. Ding, H. Zhao, and X. Qi, “Paconv: Position adaptive convolution with dynamic kernel assembling on point clouds,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog., 2021, pp. 3173–3182

    2021

  74. [82]

    Imagenet large scale visual recognition challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al., “Imagenet large scale visual recognition challenge,” Int. J. Comput. Vis. , vol. 115, pp. 211–252, 2015

    2015

  75. [83]

    Imagebind: One embedding space to bind them all,

    R. Girdhar, A. El-Nouby, Z. Liu, M. Singh, K. V . Alwala, A. Joulin, and I. Misra, “Imagebind: One embedding space to bind them all,” in IEEE/CVF Conf. Comput. Vis. Pattern Recog. , 2023, pp. 15 180– 15 190

    2023

  76. [84]

    Training data-efficient image transformers & distilla- tion through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, “Training data-efficient image transformers & distilla- tion through attention,” in Int. Conf. Mach. Learn., 2021, pp. 10 347– 10 357

    2021

  77. [85]

    Roberta: A robustly optimized bert pretraining approach,

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” ArXiv, vol. abs/1907.11692, 2019. [Online]. Available: https://api. semanticscholar.org/CorpusID:198953378 SUBMIT...

    1907 arXiv

  78. [2024]

    His current re- search directions are computer vision and 3D point cloud analysis

    He is currently working toward the Mphil degree with Guangdong Laboratory of Artificial Intelligence and Digital Economy (Shenzhen), Shenzhen University, Guangdong, China, under the supervision of Mengke Li. His current re- search directions are computer vision and 3D point cl...

    2006

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.