Pith. sign in

REVIEW 2 major objections 4 minor 109 references

Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learning

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AsymDSD claims that latent-space self-distillation, not geometry reconstruction, is the right objective for masked 3D point-cloud pretraining, and reports state-of-the-art results on ScanObjectNN.

desk verdict Solid ShapeNet-scale method and ablations; the scaled 930k-shape SOTA claim rests on an unstated pretraining split that may include test scans. read the letter →

arxiv 2506.21724 v1 pith:5AU234HN submitted 2025-06-26 cs.CV

Remco F. Leijenaar , Hamidreza Kasaei This is my paper
classification cs.CV
keywords self-supervisedlearning3Dpointcloudsmaskedmodelingself-distillationjointembeddingarchitecturerepresentationmulti-cropshapeleakage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the dominant paradigm for self-supervised 3D point-cloud learning, masked point modeling with reconstruction, points at the wrong target. Its proposal, AsymDSD, replaces geometry reconstruction with prediction in a discrete latent space: a student with an encoder-predictor architecture must reproduce the posterior of a momentum teacher over masked patches, while a second objective forces global invariance across augmented and cropped views. On the standard ShapeNet pretraining protocol, the paper reports 90.53% accuracy on ScanObjectNN's hardest split (PB_T50_RS), a 5.35-point improvement over Point-MAE with the same transformer backbone, and 94.1% on ModelNet40. Pretraining on a mixture of 930k shapes raises ScanObjectNN accuracy to 93.72%, surpassing larger autoregressive and cross-modal models. If correct, this shows that masked modeling and invariance learning can be unified in one latent-space objective without the architectural overhead and objective interference of earlier hybrids.

What carries the argument

The load-bearing mechanism is the asymmetric encoder-predictor inside a momentum-teacher joint embedding architecture. The student's encoder processes only visible patches; a wide-but-shallow predictor then builds masked-patch representations from positional queries, with self-attention between mask queries disabled so that the object's coarse shape cannot leak through positions. Both patch-level and CLS-level predictions are projected to a 4096-way discrete distribution, and centering plus sharpening on the teacher logits prevents posterior collapse. Around this core, inverse block-wise masking forces the model to infer global shape from a few localized regions, multi-mask reuses teacher targets across several masks to amortize cost, and multi-crop with two global and four local crops provides a challenging local-to-global invariance signal.

What would settle it

Pretrain AsymDSD on the Mixture dataset after deleting every ScanObjectNN and ModelNet40 point cloud, or restricted to their official training splits, then fine-tune and compare the PB_T50_RS accuracy to the reported 93.72%; if accuracy drops materially, the scaled result was inflated by the model seeing evaluation geometry during pretraining.

Watch

Extended reading notes

Core claim

The central claim is that masked point modeling for 3D data works better when the model predicts latent representations instead of raw coordinates. AsymDSD trains a joint embedding architecture by self-distillation from an EMA teacher, with two objectives computed on a discrete latent: a patch-level objective that predicts teacher posteriors for masked patches from visible context, and a global objective that aligns one view's CLS token with another view's distribution over the same latent. The paper identifies the asymmetric encoder-predictor as the component that makes this work: positional queries for masked patches are deferred to a lightweight predictor whose mask tokens do not attend to one another, so the encoder never sees the coarse global shape through the mask positions. Additional components, namely inverse block-wise masking, multi-mask sampling, and a point-cloud adaptation of multi-crop, are shown by ablations to be necessary for full performance, and removing the predictor collapses the representation. The paper reports that the combined objective beats each objective alone, and that the teacher encoder transfers better than the student.

Load-bearing premise

The scaled state-of-the-art numbers rest on the silent assumption that the Mixture pretraining set did not include the ScanObjectNN and ModelNet40 evaluation point clouds, even though those datasets are listed as components without a split restriction.

Editorial extensions

If this is right

  • Reconstruction-free masked modeling for point clouds becomes a viable alternative to masked autoencoders, potentially removing the bias toward high-frequency geometry details.
  • Disabling attention between mask queries solves the shape-leakage problem that motivated autoregressive designs, at lower compute.
  • Training can be amortized: multi-mask increases the effective batch size nearly for free, and high mask ratios let a small encoder process few visible patches.
  • Scaling works: a single-modal transformer trained on 930k shapes surpasses larger models that used supervised post-pretraining or cross-modal signals.
  • The learned representations transfer off-the-shelf, since linear probing and few-shot fine-tuning both improve, which matters for label-scarce 3D applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same dual self-distillation should transfer to other unstructured geometry representations such as meshes, implicit fields, and scene-level point clouds once the flat encoder is replaced by a hierarchical one, a direction the paper itself names as future work.
  • An implicit testable extension is that variable mask ratios could substitute for local crops, since the paper observes that block-wise masking already produces localized contexts and could remove the need for multi-crop.
  • A reader should be cautious about interpreting the scaled results until the pretraining and evaluation split is clarified: because ScanObjectNN and ModelNet40 appear in the Mixture dataset list, the 93.72% figure depends on only official training splits being used.
  • The paper's leakage argument suggests that many existing masked point-modeling methods that let mask queries attend to one another may have been reporting transfer from shape leakage rather than from better semantics; comparing their attention-distance patterns could confirm this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes AsymDSD, a self-supervised learning framework for 3D point clouds that jointly optimizes a global invariance objective over crops and a patch-level masked prediction objective in a discrete latent space. The student is an asymmetric encoder-predictor and the teacher is a momentum encoder; the authors argue that predicting latent representations rather than input geometry avoids the limitations of reconstruction-based masked point modeling. Experiments under the standard ShapeNet pretraining protocol report 90.53% on ScanObjectNN PB_T50_RS and 94.1% on ModelNet40, and a scaled pretraining run on a composite Mixture dataset of 930k shapes reports 93.72% on ScanObjectNN and 94.7% on ModelNet40. The paper includes extensive ablations of masking strategy, predictor design, multi-mask, and multi-crop, plus entropy analyses of collapse dynamics.

Significance. If the reported results hold, AsymDSD is a meaningful contribution to 3D self-supervised learning: it cleanly unifies masked modeling and invariance learning in latent space, provides an efficient asymmetric architecture, and includes unusually thorough ablations that isolate the role of each component. The derivation of the losses in Eqs. (1)-(5) is clear, the release of code and detailed hyperparameters are strengths, and the ShapeNet-pretrained experiments are internally consistent. However, the central scaled-pretraining claim is currently not verifiable as written: the Mixture pretraining set appears to contain the two evaluation datasets, and the paper never states that only their official training splits were used. The scaled SOTA claim therefore needs either explicit verification of a clean protocol or a rerun before the paper's main promise can be accepted.

major comments (2)
  1. [Appendix C.1 / Table 7; Section 4.2 / Table 4] The scaled pretraining protocol is not clean as described. Appendix C.1 defines the Mixture dataset and Table 7 explicitly lists ScanObjectNN (16,034 instances) and ModelNet40 (9,843 instances) among the pretraining sources, but neither the main text nor the appendix states that only the official training splits of these two datasets were used. Since the Section 4.2 evaluation fine-tunes and tests on the standard test splits of ScanObjectNN and ModelNet40, including the test geometries in the 930k-shape pretraining set would directly leak evaluation information into the pretraining phase. Under the stated protocol, the 93.72% (ScanObjectNN PB_T50_RS) and 94.7% (ModelNet40) results in Table 4, and the abstract's claim of 'further improves to 93.72% ... surpassing prior methods,' are not supported. The authors must either explicitly document and verify the train-only split restriction or rerun the scaled experiments with an uncontaminated pretraining set.
  2. [Appendix C.1, Table 7; Section 4.2; Abstract] The reported Mixture composition is internally inconsistent and needs correction. The ten non-Objaverse entries in Table 7 sum to 131,200 instances, not the stated 133,668, and 131,200 + 797,084 = 928,284, not 930,752. In addition, Section 4.1 states that ShapeNet pretraining uses ShapeNetCore with 41,952 models, while Table 7 lists ShapeNetCore v2 with 52,470 instances; the paper should explain whether the Mixture uses a different ShapeNetCore subset or one of these numbers is a typo. Because the abstract and Section 4.2 advertise '930k shapes,' the dataset manifest and counts must be corrected and made precise.
minor comments (4)
  1. [Section 1 and Section 3.1] There are several typos that should be corrected, including 'unfies' for 'unifies' and 'innvolves' for 'involves'; a careful proofread is needed.
  2. [Appendix C.1] The statement that 'all objects were rotated to their natural upright position in a shared reference frame' is vague; please specify the reference frame and how the upright orientation was determined for scanned objects.
  3. [Table 4 and Appendix C.3] No standard deviations or number of runs are reported for the scaled fine-tuning results, and the checklist itself acknowledges that no statistical significance testing was performed; at minimum, the number of runs behind Table 4 should be stated.
  4. [Figure 8] The full pipeline diagram is very dense and difficult to read; adding clearer separation between the teacher and student streams and labeling the loss computation points would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the self-distillation losses and reported benchmark accuracies are independent, and the flagged Mixture-pretraining overlap is a data-leakage risk, not an equation-level circularity.

full rationale

The paper's derivation chain is empirical rather than formal: AsymDSD optimizes standard cross-entropy/KL self-distillation objectives between a student and an EMA teacher (Eqs. 1-5), and the claimed results are accuracies measured on held-out downstream classification and segmentation tasks. No benchmark accuracy, ScanObjectNN score, or ModelNet40 number is used as a training target, a fitted parameter, or a term in any loss; the method is not defined in terms of the datasets it evaluates on. The teacher's latent targets are produced by the momentum encoder from the student's own weights, which is the standard self-distillation setup and does not make the downstream result tautological. Hyperparameters were chosen by hand or by ablations, not by maximizing the reported test numbers. The paper contains no load-bearing self-citation: the cited methodological anchors (DINO, iBOT, data2vec, I-JEPA, Point-MAE) are external prior work, and no uniqueness theorem or prior result by the same authors is invoked to force the design. The most serious concern raised by the reader—that the Mixture pretraining set in Table 7 lists ScanObjectNN and ModelNet40 without explicitly restricting to official training splits—is a potential evaluation-protocol leak that could inflate the scaled results, but it is not circularity: even if full datasets were used, that would be a contamination/reproducibility defect, not a reduction of the reported accuracy to the method's own fitted inputs. Under the required standard of quoting a specific equation-level reduction, none exists, so the appropriate finding is no significant circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claims rest on empirical SSL assumptions and on hand-chosen hyperparameters rather than on a formal derivation. The method introduces no new physical or formal entities; its 'predictor' and 'mask token' are learned parameters, not independently evidenced postulates.

free parameters (7)
  • Discrete latent size (Ntok) = 4096
    Projection head output dimension for both CLS and patch objectives; hand-chosen capacity for the discrete latent (Table 6c).
  • Teacher/student temperatures = tau_s=0.1; tau_t_CLS=0.04-0.07; tau_t_patch=0.05-0.07
    Control sharpness of the distillation targets and prevent collapse (Table 6b, Section 3.1).
  • Masking ratio and block size = Mr=0.7; Bs=6; Ar=0.1
    Inverse block-wise masking configuration for global crops; chosen via ablations (Table 5b, Appendix B.2).
  • Multi-mask count = 4
    Number of masks per global crop to reuse teacher targets; trades memory/throughput (Table 5d).
  • Multi-crop configuration = 2 global, 4 local; global c in [0.4,1.0], local c in [0.05,0.4]
    Adaptation of 2D multi-crop to point clouds; local crops contain one-quarter the points of global crops (Section 3.1, Table 6a).
  • EMA schedule = cosine 0.995 to 1.0
    Momentum teacher decay; standard in self-distillation (Table 6b).
  • KoLeo scale = 0.01
    Weight for batch diversity regularization (Table 6b).
assumptions (4)
  • domain assumption Cropping, rotation, and anisotropic scaling augmentations preserve the semantic category of a point cloud
    The global invariance objective (Eqs. 1-3) assumes different views of a shape share a high-level meaning; if an augmentation destroys the object identity, the CLS objective is mis-specified (Section 3.1).
  • domain assumption A momentum-EMA teacher provides stable, non-collapsing targets for both the global and patch objectives
    The framework replaces the unknown teacher posterior with an EMA of the student; centering and sharpening are added to avoid collapse, but the basic premise that EMA targets are useful is assumed (Section 3.1, Appendix D.1).
  • domain assumption The discrete latent over 4096 tokens supports semantically meaningful prediction of masked regions
    The MPM objective (Eq. 5) is a cross-entropy over this latent; its semantic utility is demonstrated empirically, not derived (Section 3.2).
  • domain assumption Inverse block-wise masking with randomly flipped bits to reach the target ratio preserves difficult-enough prediction tasks
    The masking strategy assumes local visible context is sufficient to predict the semantic class of masked patches without trivial leakage (Section 3.2, Appendix B.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learning." pith.science (2026). https://pith.science/paper/5AU234HN

@misc{pith2026250621724,
  author       = {Pith},
  title        = {Pith review of: Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5AU234HN}},
  note         = {Machine review of arXiv:2506.21724}
}
read the original abstract

Learning semantically meaningful representations from unstructured 3D point clouds remains a central challenge in computer vision, especially in the absence of large-scale labeled datasets. While masked point modeling (MPM) is widely used in self-supervised 3D learning, its reconstruction-based objective can limit its ability to capture high-level semantics. We propose AsymDSD, an Asymmetric Dual Self-Distillation framework that unifies masked modeling and invariance learning through prediction in the latent space rather than the input space. AsymDSD builds on a joint embedding architecture and introduces several key design choices: an efficient asymmetric setup, disabling attention between masked queries to prevent shape leakage, multi-mask sampling, and a point cloud adaptation of multi-crop. AsymDSD achieves state-of-the-art results on ScanObjectNN (90.53%) and further improves to 93.72% when pretrained on 930k shapes, surpassing prior methods.

Figures

Figures reproduced from arXiv: 2506.21724 by the authors.

Figure 1
Figure 1. A depiction of the size of some well-known predominantly object-centered datasets in the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. High level overview of AsymDSD. The diagram highlights the asymmetry between the teacher and student networks, and shows the distillation of knowledge from the momentum encoded (EMA) teacher on both a cross-view global (CLS) and same-view patch level. The student features an efficient design manifested by its deep but narrow encoder, and its wide but shallow predictor. To assess scalability, we pretrain AsymDSD on a… view at source ↗
Figure 3
Figure 3. Local and global crops. one-quarter the points of global crops. This sets up a challenging objective that encourages learning globally consistent representations from highly localized views. Particularly, we define the multi-crop loss over the full set of crops V =  x g 1 , x g 2 , xl 1 , . . . , xl NL [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Leakage of the coarse shape via the positions of masked patches. f s θ (x˜, cM) ≜  h proj ω ◦ g pred ψ  f enc ϕ (x˜), cM  , (6) f t θ ′ (x) ≜  h proj ω′ ◦ f enc ϕ′  (x), (7) where f enc is a contextualizing encoder processing the visible context x˜; g pred a predi…
Figure 6
Figure 6. Figure 6: The losses of AsymDSD with multi￾mask and multi-crop with both global (red and green) and local (blue) crops. To address this, we implement inverse block-wise masking (Fig. 5b), which retains only a few small contiguous regions. This forces the model to infer the globa…
Figure 7
Figure 7. Figure 7: The average attention distance per attention head across the depth of the encoder. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Overview of the processing pipeline for AsymDSD for a single point cloud through both [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 8
Figure 8. Figure 8: A.1.3 Patch Embedding The obtained patches XP are themselves small point clouds. Accordingly, before further processing, they must be projected to an embedding space. This embedding process effectively boils down to the compression of the point cloud into a single feat…
Figure 9
Figure 9. Figure 9: Building blocks of the processing pipeline. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Marginal and posterior entropy, and KL divergence during training. [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Student versus teacher performance. It shows plots of the overall accuracy on the hardest [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Zero-shot coloring of points according to inverse distance weighted nearest patch embed [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: While it was already shown that the typical transformer encoder implementation (Fig. 13a) leads to significantly reduce performance. We also tested cross-attention with concatenation of the query token itself (Fig. 13d), but observe that this yields no significant ben…
Figure 13
Figure 13. Figure 13: Different transformer block designs for the predictor. The time complexity of the attention [PITH_FULL_IMAGE:figures/full_fig_p025_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

109 extracted references · 36 canonical work pages

  1. [1]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  2. [2]

    Autoencoders as cross-modal teachers: Can pretrained 2d image transformers help 3d representation learning? arXiv preprint arXiv:2212.08320, 2022

    Runpei Dong, Zekun Qi, Linfeng Zhang, Junbo Zhang, Jianjian Sun, Zheng Ge, Li Yi, and Kaisheng Ma. Autoencoders as cross-modal teachers: Can pretrained 2d image transformers help 3d representation learning? arXiv preprint arXiv:2212.08320, 2022

    arXiv 2022

  3. [3]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017

    2017

  4. [4]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13142– 13153, 2023

    2023

  5. [5]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Information Processing Systems, 36, 2024

    2024

  6. [6]

    Self-supervised learning for pre-training 3d point clouds: A survey

    Ben Fei, Weidong Yang, Liwen Liu, Tianyue Luo, Rui Zhang, Yixuan Li, and Ying He. Self-supervised learning for pre-training 3d point clouds: A survey. arXiv preprint arXiv:2305.04691, 2023

    arXiv 2023

  7. [7]

    Bert: Pre-training of deep bidirec- tional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirec- tional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

  8. [8]

    Generative pretraining from pixels

    Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International conference on machine learning, pages 1691–1703. PMLR, 2020

    2020

Show all 109 references
  1. [9]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural i...

    2020

  2. [10]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020

    2020

  3. [11]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021

    2021

  4. [12]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022

    2022

  5. [13]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  6. [14]

    Self-supervised pretraining of visual features in the wild

    Priya Goyal, Mathilde Caron, Benjamin Lefaudeux, Min Xu, Pengchao Wang, Vivek Pai, Mannat Singh, Vitaliy Liptchinsky, Ishan Misra, Armand Joulin, et al. Self-supervised pretraining of visual features in the wild. arXiv preprint arXiv:2103.01988, 2021. 10

    2021 arXiv

  7. [15]

    The effectiveness of mae pre-pretraining for billion-scale pretraining

    Mannat Singh, Quentin Duval, Kalyan Vasudev Alwala, Haoqi Fan, Vaibhav Aggarwal, Aaron Adcock, Armand Joulin, Piotr Dollár, Christoph Feichtenhofer, Ross Girshick, et al. The effectiveness of mae pre-pretraining for billion-scale pretraining. In Proceedings of the IEEE/CVF Int...

    2023

  8. [16]

    Scaling language-free visual representation learning

    David Fan, Shengbang Tong, Jiachen Zhu, Koustuv Sinha, Zhuang Liu, Xinlei Chen, Michael Rabbat, Nicolas Ballas, Yann LeCun, Amir Bar, et al. Scaling language-free visual representation learning. arXiv preprint arXiv:2504.01017, 2025

    2025 arXiv

  9. [17]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  10. [18]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  11. [19]

    Masked siamese networks for label-efficient learning

    Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Mike Rabbat, and Nicolas Ballas. Masked siamese networks for label-efficient learning. In European Conference on Computer Vision, pages 456–473. Springer, 2022

    2022

  12. [20]

    Unsupervised point cloud pre-training via occlusion completion

    Hanchen Wang, Qi Liu, Xiangyu Yue, Joan Lasenby, and Matt J Kusner. Unsupervised point cloud pre-training via occlusion completion. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9782–9792, 2021

    2021

  13. [21]

    Point-bert: Pre-training 3d point cloud transformers with masked point modeling

    Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19313–19322, 2022

    2022

  14. [22]

    Masked autoencoders for point cloud self-supervised learning

    Yatian Pang, Wenxiao Wang, Francis EH Tay, Wei Liu, Yonghong Tian, and Li Yuan. Masked autoencoders for point cloud self-supervised learning. In European conference on computer vision, pages 604–621. Springer, 2022

    2022

  15. [23]

    Point-m2ae: multi-scale masked autoencoders for hierarchical point cloud pre-training

    Renrui Zhang, Ziyu Guo, Peng Gao, Rongyao Fang, Bin Zhao, Dong Wang, Yu Qiao, and Hongsheng Li. Point-m2ae: multi-scale masked autoencoders for hierarchical point cloud pre-training. Advances in neural information processing systems, 35:27061–27074, 2022

    2022

  16. [24]

    Pointgpt: Auto-regressively generative pre-training from point clouds

    Guangyan Chen, Meiling Wang, Yi Yang, Kai Yu, Li Yuan, and Yufeng Yue. Pointgpt: Auto-regressively generative pre-training from point clouds. arXiv preprint arXiv:2305.11487, 2023

    2023 arXiv

  17. [25]

    Towards compact 3d representations via point feature enhancement masked autoencoders

    Yaohua Zha, Huizhen Ji, Jinmin Li, Rongsheng Li, Tao Dai, Bin Chen, Zhi Wang, and Shu-Tao Xia. Towards compact 3d representations via point feature enhancement masked autoencoders. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 6962–6970, 2024

    2024

  18. [26]

    Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining

    Zekun Qi, Runpei Dong, Guofan Fan, Zheng Ge, Xiangyu Zhang, Kaisheng Ma, and Li Yi. Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining. arXiv preprint arXiv:2302.02318, 2023

    2023 arXiv

  19. [27]

    Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications

    Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.05517, 2017

    2017 arXiv

  20. [28]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning , pages 8821–8831. Pmlr, 2021

    2021

  21. [29]

    Representation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  22. [30]

    Data2vec: A general framework for self-supervised learning in speech, vision and language

    Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. Data2vec: A general framework for self-supervised learning in speech, vision and language. InInternational Conference on Machine Learning, pages 1298–1312. PMLR, 2022

    2022

  23. [31]

    Efficient self-supervised learning with contextualized target representations for vision, speech and language

    Alexei Baevski, Arun Babu, Wei-Ning Hsu, and Michael Auli. Efficient self-supervised learning with contextualized target representations for vision, speech and language. In International Conference on Machine Learning, pages 1416–1429. PMLR, 2023. 11

    2023

  24. [32]

    Self-supervised learning from images with a joint-embedding predictive architecture

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2023

  25. [33]

    ibot: Image bert pre-training with online tokenizer

    Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832, 2021

    2021 arXiv

  26. [34]

    Point2vec for self-supervised representation learning on point clouds

    Karim Abou Zeid, Jonas Schult, Alexander Hermans, and Bastian Leibe. Point2vec for self-supervised representation learning on point clouds. arXiv preprint arXiv:2303.16570, 2023

    2023

  27. [35]

    Unsu- pervised learning of visual features by contrasting cluster assignments

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsu- pervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912–9924, 2020

    2020

  28. [36]

    Shapenet: An information-rich 3d model repository

    Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015

    2015 arXiv

  29. [37]

    Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data

    Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In Proceedings of the IEEE/CVF international conference on computer vision, pages 15...

    2019

  30. [38]

    3d shapenets: A deep representation for volumetric shapes

    Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912–1920, 2015

    1912

  31. [39]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. arxiv e-prints, art. arXiv preprint arXiv:1911.05722, 2019

    1911 arXiv

  32. [40]

    Exploring simple siamese representation learning

    Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750–15758, 2021

    2021

  33. [41]

    Semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples

    Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Armand Joulin, Nicolas Ballas, and Michael Rabbat. Semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples. In Proceedings of the IEEE/CVF International Co...

    2021

  34. [42]

    Barlow twins: Self-supervised learning via redundancy reduction

    Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International conference on machine learning, pages 12310–12320. PMLR, 2021

    2021

  35. [43]

    Vicreg: Variance-invariance-covariance regularization for self-supervised learning

    Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. arXiv preprint arXiv:2105.04906, 2021

    2021 arXiv

  36. [44]

    Extracting and composing robust features with denoising autoencoders

    Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pages 1096–1103, 2008

    2008

  37. [45]

    Simmim: A simple framework for masked image modeling

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9653–9663, 2022

    2022

  38. [46]

    Beit: Bert pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021

    2021 arXiv

  39. [47]

    Discrete variational autoencoders

    Jason Tyler Rolfe. Discrete variational autoencoders. arXiv preprint arXiv:1609.02200, 2016

    2016 arXiv

  40. [48]

    Pointcontrast: Unsupervised pre-training for 3d point cloud understanding

    Saining Xie, Jiatao Gu, Demi Guo, Charles R Qi, Leonidas Guibas, and Or Litany. Pointcontrast: Unsupervised pre-training for 3d point cloud understanding. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16 , pages ...

    2020

  41. [49]

    Self-supervised pretraining of 3d features on any point-cloud

    Zaiwei Zhang, Rohit Girdhar, Armand Joulin, and Ishan Misra. Self-supervised pretraining of 3d features on any point-cloud. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10252–10263, 2021. 12

    2021

  42. [50]

    Crosspoint: Self-supervised cross-modal contrastive learning for 3d point cloud understanding

    Mohamed Afham, Isuru Dissanayake, Dinithi Dissanayake, Amaya Dharmasiri, Kanchana Thilakarathna, and Ranga Rodrigo. Crosspoint: Self-supervised cross-modal contrastive learning for 3d point cloud understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

    2022

  43. [51]

    Spatio-temporal self-supervised representation learning for 3d point clouds

    Siyuan Huang, Yichen Xie, Song-Chun Zhu, and Yixin Zhu. Spatio-temporal self-supervised representation learning for 3d point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6535–6545, 2021

    2021

  44. [52]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv...

    2010 arXiv

  45. [53]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  46. [54]

    On mutual information maximization for representation learning

    Michael Tschannen, Josip Djolonga, Paul K Rubenstein, Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. arXiv preprint arXiv:1907.13625, 2019

    1907 arXiv

  47. [55]

    Self-organization in a perceptual network

    Ralph Linsker. Self-organization in a perceptual network. Computer, 21(3):105–117, 1988

    1988

  48. [56]

    Self-supervised learning for domain adaptation on point clouds

    Idan Achituve, Haggai Maron, and Gal Chechik. Self-supervised learning for domain adaptation on point clouds. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 123–133, 2021

    2021

  49. [57]

    Spreading vectors for similarity search

    Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, and Hervé Jégou. Spreading vectors for similarity search. arXiv preprint arXiv:1806.03198, 2018

    2018 arXiv

  50. [58]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  51. [59]

    Understanding self-supervised learning dynamics without contrastive pairs

    Yuandong Tian, Xinlei Chen, and Surya Ganguli. Understanding self-supervised learning dynamics without contrastive pairs. In International Conference on Machine Learning, pages 10268–10278. PMLR, 2021

    2021

  52. [60]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pages 10347–10357. PMLR, 2021

    2021

  53. [61]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    CR Qi, H Su, K Mo, and LJ Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. cvpr 2017. arXiv preprint arXiv:1612.00593, 2016

    2017 arXiv

  54. [62]

    Rethinking network design and local geometry in point cloud: A simple residual mlp framework

    Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. Rethinking network design and local geometry in point cloud: A simple residual mlp framework. arXiv preprint arXiv:2202.07123, 2022

    2022 arXiv

  55. [63]

    Pointnext: Revisiting pointnet++ with improved training and scaling strategies

    Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in Neural Information Processing Systems, 35:23192–23204, 2022

    2022

  56. [64]

    Masked discrimination for self-supervised learning on point clouds

    Haotian Liu, Mu Cai, and Yong Jae Lee. Masked discrimination for self-supervised learning on point clouds. In European Conference on Computer Vision, pages 657–675. Springer, 2022

    2022

  57. [65]

    Regress before construct: Regress autoencoder for point cloud self-supervised learning

    Yang Liu, Chen Chen, Can Wang, Xulin King, and Mengyuan Liu. Regress before construct: Regress autoencoder for point cloud self-supervised learning. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1738–1749, 2023

    2023

  58. [66]

    Pointmamba: A simple state space model for point cloud analysis

    Dingkang Liang, Xin Zhou, Wei Xu, Xingkui Zhu, Zhikang Zou, Xiaoqing Ye, Xiao Tan, and Xiang Bai. Pointmamba: A simple state space model for point cloud analysis. arXiv preprint arXiv:2402.10739, 2024

    2024 arXiv

  59. [67]

    Deep networks with stochastic depth

    Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger. Deep networks with stochastic depth. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 646–661. Springer, 2016

    2016

  60. [68]

    Self-supervised few-shot learning on point clouds

    Charu Sharma and Manohar Kaul. Self-supervised few-shot learning on point clouds. Advances in Neural Information Processing Systems, 33:7212–7221, 2020

    2020

  61. [69]

    A scalable active framework for region annotation in 3d shape collections

    Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics (ToG), 35(6):1–12, 2016. 13

    2016

  62. [70]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017

    2017

  63. [71]

    Shapellm: Universal 3d object understanding for embodied interaction

    Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. In European Conference on Computer Vision, pages 214–238. Springer, 2024

    2024

  64. [72]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  65. [73]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  66. [74]

    Swin3d: A pretrained transformer backbone for 3d indoor scene understanding

    Yu-Qi Yang, Yu-Xiao Guo, Jian-Yu Xiong, Yang Liu, Hao Pan, Peng-Shuai Wang, Xin Tong, and Baining Guo. Swin3d: A pretrained transformer backbone for 3d indoor scene understanding. arXiv preprint arXiv:2304.06906, 2023

    2023 arXiv

  67. [75]

    Octformer: Octree-based transformers for 3d point clouds

    Peng-Shuai Wang. Octformer: Octree-based transformers for 3d point clouds. ACM Transactions on Graphics (TOG), 42(4):1–11, 2023

    2023

  68. [76]

    Point transformer v3: Simpler, faster, stronger

    Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler, faster, stronger. arXiv preprint arXiv:2312.10035, 2023

    2023 arXiv

  69. [77]

    Mixed precision training

    Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXiv:1710.03740, 2017

    arXiv

  70. [78]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  71. [79]

    Root mean square layer normalization

    Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019

    2019

  72. [80]

    Gaussian error linear units (gelus)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016

    2016 arXiv

  73. [81]

    3d semantic parsing of large-scale indoor spaces

    Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3d semantic parsing of large-scale indoor spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1534–1543, 2016

    2016

  74. [82]

    Sun rgb-d: A rgb-d scene understanding benchmark suite

    Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 567–576, 2015

    2015

  75. [83]

    Lvis: A dataset for large vocabulary instance segmentation

    Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356–5364, 2019

    2019

  76. [84]

    Openshape: Scaling up 3d shape representation towards open-world understanding

    Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open-world understanding. Advances in neural information processing systems, 36:44860–44879, 2023

    2023

  77. [85]

    Uni3d: Exploring unified 3d representation at scale

    Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. arXiv preprint arXiv:2310.06773, 2023

    2023 arXiv

  78. [86]

    3d-future: 3d furniture shape with texture

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d furniture shape with texture. International Journal of Computer Vision, 129:3313–3337, 2021

    2021

  79. [87]

    Abo: Dataset and benchmarks for real-world 3d object understanding

    Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d object understanding. In Proceedings of the IEEE/CVF conference on com...

    2022

  80. [88]

    Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation

    Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, et al. Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. In Proceedings of the IEEE/CVF Conference on Computer ...

    2023

  81. [89]

    Using shape to categorize: Low-shot learning with an explicit shape bias

    Stefan Stojanov, Anh Thai, and James M Rehg. Using shape to categorize: Low-shot learning with an explicit shape bias. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1798–1808, 2021

    2021

  82. [90]

    Google scanned objects: A high-quality dataset of 3d scanned household items

    Anthony G Francis, Brandon Kinman, Krista Ann Reymann, Laura Downs, Nathan Koenig, Ryan M Hickman, Thomas B McHugh, and Vincent Olivier Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned household items. 2022

    2022

  83. [91]

    Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results

    Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017

    2017

  84. [92]

    Fast r-cnn

    Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440–1448, 2015. 15 A Overall Model Pipeline Although many dedicated models architectures for point cloud data have been devised, there is a lack of a unified architecture ...

    2015

  85. [93]

    For more details of the design refer to the indicated figures

    in GiB; and It/s the throughput in iterations per second. For more details of the design refer to the indicated figures. Method Fig. MB Attention Mem. It/s ModelNet40 ScanObjectNN Self Cross SVM FFt SVM FFt (MPM-S) 13c × × c 16.2 4.80 93.31 94.04 81.06 88.58 13d × × c+s 17.5 4...

  86. [94]

    3 and Sec

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: The claims are substantiated in the Sec. 3 and Sec. 4. Guidelines: • The answer NA means that the abstract and introdu...

  87. [95]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: There is a dedication section with limitations in Sec. 5. Guidelines: • The answer NA means that the paper has no limitation while the answer No mean...

  88. [96]

    Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] 26 Justification: The paper does not include theoretical results. Guidelines: • The answer NA means that the...

  89. [97]

    Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and da...

  90. [98]

    We include code with configurations to reproduce results

    Open access to data and code 27 Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: Data is publicly availab...

  91. [99]

    Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: The general experimental sett...

  92. [100]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [No] Justification: We do not test for statistical significance, but do...

  93. [101]

    Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: We disclose the compute...

  94. [102]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: The conducted research is conform the NeurIPS Code of Ethics. Guidelines: • The...

  95. [103]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [NA] Justification: There is no direct societal impact, as we propose a novel learning framework for 3D point clouds. Guidelin...

  96. [104]

    Guidelines: • The answer NA means that the paper poses no such risks

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: There is no suc...

  97. [105]

    Our work does not redistribute existing assets

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We cite all orig...

  98. [106]

    Guidelines: • The answer NA means that the paper does not release new assets

    New assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: We do not release new assets. Guidelines: • The answer NA means that the paper does not release new assets. • Researc...

  99. [107]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? A...

  100. [108]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

  101. [109]

    Answer: [NA] Justification: The core method development does not involve LLMs

    Declaration of LLM usage Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the ...

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.