Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

SRKD: Towards Efficient 3D Point Cloud Segmentation via Structure- and Relation-aware Knowledge Distillation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SRKD transfers geometric structure from a frozen 101.4M-parameter teacher to an 11.6M-parameter student by aligning affinity and cross-sample similarity matrices, matching the teacher's 77.9 mIoU on ScanNet with less than one-eighth the…

desk verdict The headline SOTA claim hangs on a baseline that differs by 5.9 mIoU between Table 1 and Table 5; the method is plausible but the numbers need reconciliation before trust. read the letter →

arxiv 2506.17290 v1 pith:33T47T7R submitted 2025-06-16 cs.CV

Yuqi Li , Junhao Dong , Zeyu Dong , Chuanguang Yang , Zhulin An , Yongjun Xu This is my paper · ORCID
classification cs.CV
keywords pointcloudsemanticsegmentationknowledgedistillationaffinitymatrixcross-samplegeometryalignmentrelation-awaremodelcompressionLiDARScanNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a lightweight 3D point-cloud segmentation model can reach the accuracy of a transformer model roughly nine times its size, provided the distillation transfers geometric structure rather than only per-point predictions. Its SRKD framework does this by aligning pairwise similarity matrices between student and teacher features, both within a scene (point, voxel, and channel levels) and across scenes in a mini-batch, alongside the usual soft-label and ground-truth losses. On the ScanNet benchmark the 11.6M-parameter student reaches 77.9 mIoU, matching the 101.4M-parameter teacher and beating the non-distilled baseline by 1.2 points, with larger gains on the outdoor nuScenes benchmark (80.5 versus 77.9 for the baseline). If correct, the recipe matters because it lets near-teacher accuracy run on devices that cannot host a 100M-parameter model.

What carries the argument

The machinery is a family of pairwise similarity matrices that carry the teacher's geometric relations. Within each supervoxel, an affinity matrix $D(i,j,w_i)=w_i\|F_i-F_j\|_2^2$ scores every pair of point features (and likewise voxel features), with a class-balanced sampling weight $w_i=\frac{\tau_{\text{class}}}{N_v}\cdot\frac{D_i}{R}$ that up-weights rare classes; the student minimizes the $\ell^2$ distance between its affinity matrices and the teacher's. Across the mini-batch, a cross-sample similarity matrix $M_{ij}=F_iF_j^T$ between point clouds $i$ and $j$ is computed on L2-normalized features, and the student's matrix is aligned to the teacher's row-by-row through temperature-scaled KL divergence (Eqs. 8-9). These are combined with logit-level KL divergence and channel-wise activation KL into the total objective of Eq. (10), a weighted sum of the task cross-entropy and five distillation terms, so geometric, relational, and semantic signals enter the student's training jointly.

What would settle it

Retrain the full pipeline with the rows of the teacher's cross-sample similarity matrix $M^t_{ij}$ randomly permuted before the KL alignment, so student row i is matched against an unrelated teacher row. If the student's ScanNet mIoU stays near 77.9, the loss is only acting as a regularizer and the correspondence assumption is void; if the gain over the 76.7 baseline disappears, the method truly depends on matched cross-sample structure.

Watch

Extended reading notes

Core claim

The author's claim, stated in its strongest form, is that the student can match the teacher: an 11.6M-parameter PTv3-based student distilled from the frozen 101.4M-parameter CDSegNet teacher reaches 77.9 mIoU on ScanNet, identical to the teacher, and 80.5 mIoU on nuScenes, 0.7 points below the teacher and 2.6 above the same student without distillation. The mechanism credited for this is relation-level transfer: instead of matching only the teacher's output logits, the student is forced to reproduce the teacher's pairwise feature-affinity matrices at point and voxel levels, its channel-wise activations, and the paper's distinctive step, a cross-sample similarity matrix computed between every pair of point clouds in the mini-batch and aligned row-wise by KL divergence. The paper argues that unordered point clouds cannot be aligned point-to-point across scenes, so aligning the geometry of the relations between scenes is what carries generalized structural knowledge from teacher to student.

Load-bearing premise

The load-bearing premise is that aligning the point rows of two point clouds that have no point-to-point correspondence, since the cross-sample geometry loss treats the i-th point of one scene as if it matched the i-th point of another, transfers the teacher's reusable geometric structure rather than fitting noise from arbitrary point ordering or feature scale.

Editorial extensions

If this is right

  • Near-teacher accuracy at under one-eighth the parameters: on ScanNet the 11.6M-parameter student reaches 77.9 mIoU, the same as the 101.4M-parameter teacher, while using 3.5 GB of inference memory and 50 s per scene versus the teacher's 112 s.
  • The student inherits robustness traits of the teacher: under Gaussian feature noise at $\tau=0.5$ it scores 57.1 mIoU, slightly above the teacher's 57.0, and it beats the non-distilled baseline at every noise level the paper tested.
  • Distillation compensates for scarce labels: trained on 5% of ScanNet scenes the student reaches 57.1 mIoU, versus 46.2 for the teacher and 34.0 for the baseline trained on the same subsample.
  • The components are additive in the paper's ablation: the channel-reduced baseline (70.8) rises to 72.3 with logit-KL distillation, to 74.3 with cross-sample geometry alignment added, and to 75.0 with affinity-matrix alignment added, showing both proposed modules contribute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the method works as described, the same recipe should transfer other teacher properties that are costly to obtain directly, such as temporal consistency, calibration, or robustness to sensor dropout, since the student already inherits the teacher's noise resilience without any training signal aimed at noise.
  • The cross-sample alignment is an invariance constraint on unordered scenes, so a natural extension is to make it permutation-invariant explicitly, for example by sorting similarity-matrix rows under a canonical feature ordering, which would let the loss work at batch size 1 and across scenes with different point densities.
  • The nuScenes per-class table shows the student's largest gains over the baseline in sparse classes such as trucks and motorcycles, suggesting that weighting the affinity loss by class frequency could push long-tail accuracy even higher.
  • The paper's own timing table shows the affinity-matrix module roughly triples training time (64 h versus 23 h on ScanNet) even though inference is unchanged, so applications that retrain often should weigh that training cost against the parameter savings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents SRKD, a knowledge distillation framework for 3D point cloud semantic segmentation. A frozen CDSegNet teacher (>100M parameters) is used to supervise a channel-reduced PTv3 student (11.6M parameters). The training objective combines per-point KL divergence on semantic logits, L2 matching of point-level and voxel-level affinity matrices, a channel-wise activation alignment term, and a 'cross-sample mini-batch geometry distillation' loss that aligns row-wise distributions of pairwise similarity matrices computed between all pairs of point clouds in a mini-batch. Experiments on ScanNet report 77.9 mIoU for the student, matching CDSegNet, and experiments on nuScenes report 80.5 mIoU. The paper claims state-of-the-art performance with an order-of-magnitude reduction in parameters.

Significance. If the reported results are accurate, SRKD would provide a practical recipe for compressing large 3D segmentation transformers: a lightweight student with 11.6M parameters matching a 101.4M-parameter teacher on ScanNet, with additional evidence of robustness to noise and subsampled training data. The cross-sample relational distillation idea is novel for point clouds and the authors provide a public code link. However, the empirical core of the paper is currently compromised by an unreconciled baseline discrepancy between the main comparison and the ablation study, and several methodological definitions are internally inconsistent. For these reasons the significance can only be assessed after major revision.

major comments (4)
  1. [Sec. 4.2 vs Sec. 4.5 (Tables 1 and 5)] Table 1 and Table 5 report the same no-distillation baseline (the channel-reduced PTv3 student) as 76.7 mIoU with 84.2 mAcc and 91.6 allAcc, and as 70.8 mIoU with 76.4 mAcc and 87.5 allAcc, respectively. The text does not state any difference in training protocol between these two tables. With the Table 1 baseline, the full SRKD model in Table 5 (75.0 mIoU) is worse than the baseline it is supposed to improve; with the Table 5 baseline, the claimed component gains do not reproduce the Table 1 result (77.9). This 5.9-point inconsistency is load-bearing for the headline claim and must be reconciled with corrected numbers or an explicit statement of the differing experimental conditions.
  2. [Sec. 3.1, Eq. (2)] Eq. (2): the expression KL(σ(Z_i^s)/T || σ(Z_i^t)/T) is not a valid Kullback-Leibler divergence for T≠1, because dividing a softmax output by T removes normalization; with T=2 each argument sums to 1/2. The standard softened-target formulation uses softmax(z/T). Since T=2 is adopted in the experiments and L_kd is a central loss, this technical error must be corrected.
  3. [Sec. 3.2 and Algorithm 1] The loss definitions are inconsistent and partly malformed. Eq. (7) for L_amra^c has unmatched parentheses, an undefined summation index i, and mixes point and voxel terms in a single expression. Algorithm 1 defines L_amrap, L_amrav, L_amrac, and L_batch-GD, while the text uses L_amra^p, L_amra^v, L_amra^c, and the total loss in Eq. (10) uses λ_p, λ_v, λ_c, λ_batch-GD without a unique mapping. Section 3.4 refers to 'Equation (3)' and 'Equation (5)' for the cross-sample and affinity components, which are actually Eqs. (9) and (5)–(6). In addition, Eq. (8) writes the left-hand side as L_GD(MM_ij, S_t_ij) with undefined symbols. These inconsistencies make the training objective impossible to reproduce from the manuscript.
  4. [Sec. 3.3, Eqs. (8)–(9)] The cross-sample geometry distillation aligns row-wise distributions of M_ij, the similarity matrix between point clouds i and j in the mini-batch. Because point clouds are unordered and no correspondences are established across scenes, the meaning of a particular row of M_ij depends on the arbitrary ordering of points within cloud j. Although the teacher and student share the same input ordering, it is not established that M_ij encodes reusable geometric structure rather than order-dependent artifacts or feature-scale information. As this is the main novel component, please provide an empirical test (e.g., permuting point order within a cloud during training, or visualizing the learned similarity distributions) to demonstrate that the loss transfers geometric knowledge.
minor comments (6)
  1. [Sec. 4.2] The text says 'Table 3 presents the superior performance of SRKD' when referring to outdoor results; the outdoor results are in Table 2, and Table 3 contains the noise robustness experiments on ScanNet. Please fix the cross-reference.
  2. [Abstract and Table 2] The abstract's unqualified 'state of the art performance' is not supported on nuScenes, where SRKD (80.5 mIoU) is below CDSegNet (81.2 mIoU); please qualify the claim to ScanNet or to the efficient-student setting.
  3. [Sec. 4.4, Table 4] The Baseline row is non-monotonic across subsampling ratios (25%: 61.2, 12.5%: 64.1, 10%: 62.2); please verify the column alignment or explain the random subsampling procedure.
  4. [Sec. 4.1] The sentence describing the learning-rate schedule is incomplete ('with and .'); please provide the missing values.
  5. [Sec. 4.2] The paper reports a single run with no error bars; given that the headline improvement in Table 1 is 1.2 mIoU, a statement of variance or repeated-run statistics would substantially increase confidence.
  6. [Sec. 3.3] The name 'cross-sample mini-batch construction strategy' is misleading because the method does not construct new samples but rather applies a loss to pairwise combinations within the existing mini-batch; please clarify the terminology.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SRKD's central claims are evaluated against an external teacher (CDSegNet) and external benchmarks, with distillation losses acting as training objectives rather than as re-fitted predictions.

full rationale

The paper's derivation chain is self-contained in the relevant sense: the proposed losses (L_kd, L_amra, L_batch-GD) direct the student's features/logits toward the frozen teacher's outputs, and the final mIoU numbers on ScanNet and nuScenes are measured against held-out labels from external datasets. The relation/affinity and cross-sample losses are not fitted to the evaluation metric and then reported as predictions; they are training objectives whose effect is assessed by external benchmarks. The related-work self-citations ([14,17,19-22]) describe prior 2D KD formulations and are cited as background, not as the evidence establishing the paper's state-of-the-art claim. The unreconciled baseline values in Table 1 (76.7) and Table 5 (70.8) raise a reproducibility/correctness concern but are not a circularity: the discrepancy is between two reported measurements of the same configuration, not a case of an output being equivalent to an input by construction. No uniqueness theorem, ansatz-by-citation, or renamed-known-result pattern is present. Accordingly, no circular step is exhibited, and the appropriate score is 0.

Assumptions & free parameters 9 free parameters · 4 assumptions · 0 invented entities

The central claim rests on roughly nine hand-tuned hyperparameters and four modeling assumptions. The loss weights and temperature are tuned on the validation metric, and the voxel sampling and matrix dimension choices are not grounded in independent evidence. The key modeling premise is that point clouds from different scenes can be aligned through pairwise similarity matrices even though points have no semantic correspondence.

free parameters (9)
  • lambda_kd = 0.3
    Weight for the semantic KL distillation loss, tuned on validation.
  • lambda_p = 0.001
    Weight for the point-level affinity matrix distillation loss.
  • lambda_v = 0.001
    Weight for the voxel-level affinity matrix distillation loss.
  • lambda_c = 1000
    Weight for the channel-wise activation distillation loss.
  • lambda_batch-GD = 0.1
    Weight for the cross-sample geometric similarity loss.
  • temperature T = 2
    Temperature used in KL divergences; affects the sharpness of soft targets.
  • N (points per sample in similarity matrix) = 1024
    Number of points used to compute the cross-sample geometric similarity matrices, chosen as a memory-accuracy trade-off.
  • geometric feature dimension = 128
    Feature dimension for the geometric similarity matrix, selected by ablation in the supplementary material.
  • voxel grid and supervoxel sampling parameters = not reported
    The paper does not give concrete values for voxel resolutions R_v, A_v, H_v or the number of sampled supervoxels K, though these affect the affinity matrices.
assumptions (4)
  • domain assumption The teacher model CDSegNet's published accuracy is correct and its frozen features are reliable semantic targets for the student.
    The entire distillation relies on the teacher's predictions and features; any error in teacher training or evaluation would propagate to the student results.
  • domain assumption Point indices in teacher and student correspond after the same voxelization and sampling, so point-wise losses are well defined.
    Sections 3.2 and 3.3 implicitly assume index-wise alignment for the affinity matrices.
  • ad hoc to paper Cross-sample similarity matrices between unordered point clouds capture transferable geometric structure.
    Section 3.3 introduces CSMBGD without proving that pairwise feature similarities across different scenes carry reusable information beyond same-input feature matching.
  • standard math The KL divergence terms as implemented are valid probability alignments, with temperature applied inside the softmax.
    Section 3.1 Eq. (2) and Algorithm 1; the printed formula is unnormalized for T=2, so this assumption is only satisfied if the implementation differs from the text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SRKD: Towards Efficient 3D Point Cloud Segmentation via Structure- and Relation-aware Knowledge Distillation." pith.science (2026). https://pith.science/paper/33T47T7R

@misc{pith2026250617290,
  author       = {Pith},
  title        = {Pith review of: SRKD: Towards Efficient 3D Point Cloud Segmentation via Structure- and Relation-aware Knowledge Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/33T47T7R}},
  note         = {Machine review of arXiv:2506.17290}
}
read the original abstract

3D point cloud segmentation faces practical challenges due to the computational complexity and deployment limitations of large-scale transformer-based models. To address this, we propose a novel Structure- and Relation-aware Knowledge Distillation framework, named SRKD, that transfers rich geometric and semantic knowledge from a large frozen teacher model (>100M) to a lightweight student model (<15M). Specifically, we propose an affinity matrix-based relation alignment module, which distills structural dependencies from the teacher to the student through point-wise similarity matching, enhancing the student's capability to learn contextual interactions. Meanwhile, we introduce a cross-sample mini-batch construction strategy that enables the student to perceive stable and generalized geometric structure. This aligns across diverse point cloud instances of the teacher, rather than within a single sample. Additionally, KL divergence is applied to align semantic distributions, and ground-truth supervision further reinforces accurate segmentation. Our method achieves state of the art performance with significantly reduced model complexity, demonstrating its effectiveness and efficiency in real-world deployment scenarios. Our Code is available at https://github.com/itsnotacie/SRKD.

Figures

Figures reproduced from arXiv: 2506.17290 by the authors.

Figure 1
Figure 1. Overall Frameworkof SRKD. SRKD enables the student model to capture both coarse [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. The visualization results on ScanNet. Indoor Dataset. We first compare our method with existing state-of-the-art point cloud semantic segmentation methods. As shown in Tab. 1, our method achieves superior segmentation performance, 7 [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. The ablation study of mini-batch Sensitivity. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative results on nuScenes. From left to right: ground-truth labels, baseline(1/2PTv3) [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Where Detectors Fail: Closing the Tail-Domain Gap with Expert-Guided Mutual Distillation

    cs.CL 2026-07 conditional novelty 6.0 of 10

    EGMD reduces cross-domain bias in multimodal fake-news detection by calibrating input coherence, aligning domain statistics with a normalized expert teacher, and distilling into prototype-anchored student branches.

  2. SWA-SOP: Spatially-aware Window Attention for Semantic Occupancy Prediction in Autonomous Driving

    cs.CV 2025-06 conditional novelty 6.0 of 10

    SWA-SOP shows that sliding-window attention with per-slot spatial embeddings and a center query improves semantic occupancy prediction on LiDAR and camera inputs, but the headline benchmark numbers are second-best rat...

  3. ${C}^{3}$-GS: Learning Context-aware, Cross-dimension, Cross-scale Feature for Generalizable Gaussian Splatting

    cs.CV 2025-08 conditional novelty 4.0 of 10

    C3-GS improves generalizable Gaussian Splatting by adding coordinate-aware attention, cross-dimensional attention, and cross-scale opacity fusion to the MVSGaussian baseline.

Reference graph

Works this paper leans on

43 extracted references · 27 canonical work pages · cited by 3 Pith papers

  1. [1]

    Double-view feature fusion network for lidar semantic segmentation.Journal of Image and Graphics, 29(1):205–217, 2024

    Liujie Sun, Tengfei Zeng, Jingxing Fan, and Wenju Wang. Double-view feature fusion network for lidar semantic segmentation.Journal of Image and Graphics, 29(1):205–217, 2024

    work page 2024

  2. [2]

    3d point cloud segmentation: A survey

    Anh Nguyen and Bac Le. 3d point cloud segmentation: A survey. In2013 6th IEEE conference on robotics, automation and mechatronics (RAM), pages 225–230. IEEE, 2013

    work page 2013

  3. [3]

    Abderrazzaq Kharroubi, Rafika Hajji, Roland Billen, and Florent Poux. Classification and integration of massive 3d points clouds in a virtual reality (vr) environment.International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42(W17), 2019

    work page 2019

  4. [4]

    Pointconv: Deep convolutional networks on 3d point clouds

    Wenxuan Wu, Zhongang Qi, and Li Fuxin. Pointconv: Deep convolutional networks on 3d point clouds. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 9621–9630, 2019

    2019

  5. [5]

    Point-to-voxel knowledge distillation for lidar semantic segmentation

    Yuenan Hou, Xinge Zhu, Yuexin Ma, Chen Change Loy, and Yikang Li. Point-to-voxel knowledge distillation for lidar semantic segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8479–8488, 2022

    work page 2022

  6. [6]

    Point transformer

    Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 16259–16268, 2021

    2021

  7. [7]

    4d spatio-temporal convnets: Minkowski convolutional neural networks

    Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3075–3084, 2019

    2019

  8. [8]

    Stratified transformer for 3d point cloud segmentation

    Xin Lai, Jianhui Liu, Li Jiang, Liwei Wang, Hengshuang Zhao, Shu Liu, Xiaojuan Qi, and Jiaya Jia. Stratified transformer for 3d point cloud segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8500–8509, 2022

    2022

Show all 43 references
  1. [9]

    Octformer: Octree-based transformers for 3d point clouds.ACM Transactions on Graphics (TOG), 42(4):1–11, 2023

    Peng-Shuai Wang. Octformer: Octree-based transformers for 3d point clouds.ACM Transactions on Graphics (TOG), 42(4):1–11, 2023

    2023

  2. [10]

    Point transformer v2: Grouped vec- tor attention and partition-based pooling.Advances in Neural Information Processing Systems, 35:33330– 33342, 2022

    Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Hengshuang Zhao. Point transformer v2: Grouped vec- tor attention and partition-based pooling.Advances in Neural Information Processing Systems, 35:33330– 33342, 2022

    2022

  3. [11]

    Point transformer v3: Simpler faster stronger

    Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4840–4851, 2024

    2024

  4. [12]

    An end-to-end robust point cloud semantic segmentation network with single-step conditional diffusion models.arXiv preprint arXiv:2411.16308, 2024

    Wentao Qu, Jing Wang, YongShun Gong, Xiaoshui Huang, and Liang Xiao. An end-to-end robust point cloud semantic segmentation network with single-step conditional diffusion models.arXiv preprint arXiv:2411.16308, 2024

    2024 arXiv

  5. [13]

    Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  6. [14]

    Hierarchical self-supervised augmented knowledge distillation

    Chuanguang Yang, Zhulin An, Linhang Cai, and Yongjun Xu. Hierarchical self-supervised augmented knowledge distillation. InProceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pages 1217–1223, 2021

    2021

  7. [15]

    Fitnets: Hints for thin deep nets.arXiv preprint arXiv:1412.6550, 2014

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets.arXiv preprint arXiv:1412.6550, 2014

    2014 arXiv

  8. [16]

    Distilling knowledge via knowledge review

    Pengguang Chen, Shu Liu, Hengshuang Zhao, and Jiaya Jia. Distilling knowledge via knowledge review. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5008–5017, 2021

    2021

  9. [17]

    Clip-kd: An empirical study of clip model distillation

    Chuanguang Yang, Zhulin An, Libo Huang, Junyu Bi, Xinqiang Yu, Han Yang, Boyu Diao, and Yongjun Xu. Clip-kd: An empirical study of clip model distillation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15952–15962, 2024

    2024

  10. [18]

    Relational knowledge distillation

    Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3967–3976, 2019. 10

    2019

  11. [19]

    Mutual contrastive learning for visual representation learning

    Chuanguang Yang, Zhulin An, Linhang Cai, and Yongjun Xu. Mutual contrastive learning for visual representation learning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3045–3053, 2022

    2022

  12. [20]

    Cross-image relational knowledge distillation for semantic segmentation

    Chuanguang Yang, Helong Zhou, Zhulin An, Xue Jiang, Yongjun Xu, and Qian Zhang. Cross-image relational knowledge distillation for semantic segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12319–12328, 2022

    2022

  13. [21]

    Online knowledge distillation via mutual contrastive learning for visual recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):10212–10227, 2023

    Chuanguang Yang, Zhulin An, Helong Zhou, Fuzhen Zhuang, Yongjun Xu, and Qian Zhang. Online knowledge distillation via mutual contrastive learning for visual recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):10212–10227, 2023

    2023

  14. [22]

    Rela- tional diffusion distillation for efficient image generation

    Weilun Feng, Chuanguang Yang, Zhulin An, Libo Huang, Boyu Diao, Fei Wang, and Yongjun Xu. Rela- tional diffusion distillation for efficient image generation. InProceedings of the 32nd ACM International Conference on Multimedia, pages 205–213, 2024

    2024

  15. [23]

    Structured knowledge distillation for semantic segmentation

    Yifan Liu, Ke Chen, Chris Liu, Zengchang Qin, Zhenbo Luo, and Jingdong Wang. Structured knowledge distillation for semantic segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2604–2613, 2019

    2019

  16. [24]

    Transkd: Transformer knowledge distillation for efficient semantic segmentation

    Ruiping Liu, Kailun Yang, Alina Roitberg, Jiaming Zhang, Kunyu Peng, Huayao Liu, Yaonan Wang, and Rainer Stiefelhagen. Transkd: Transformer knowledge distillation for efficient semantic segmentation. IEEE Transactions on Intelligent Transportation Systems, 2024

    2024

  17. [25]

    Pointdistiller: Structured knowledge distillation towards efficient and compact 3d detection

    Linfeng Zhang, Runpei Dong, Hung-Shuo Tai, and Kaisheng Ma. Pointdistiller: Structured knowledge distillation towards efficient and compact 3d detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21791–21801, 2023

    2023

  18. [26]

    Balanced residual distillation learning for 3d point cloud class-incremental semantic segmentation.Expert Systems with Applications, 269:126399, 2025

    Yuanzhi Su, Siyuan Chen, and Yuan-Gen Wang. Balanced residual distillation learning for 3d point cloud class-incremental semantic segmentation.Expert Systems with Applications, 269:126399, 2025

    2025

  19. [27]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017

    2017

  20. [28]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space.Advances in neural information processing systems, 30, 2017

    2017

  21. [29]

    Kpconv: Flexible and deformable convolution for point clouds

    Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas. Kpconv: Flexible and deformable convolution for point clouds. InProceedings of the IEEE/CVF international conference on computer vision, pages 6411–6420, 2019

    2019

  22. [30]

    Large-scale point cloud semantic segmentation with superpoint graphs

    Loic Landrieu and Martin Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4558–4567, 2018

    2018

  23. [31]

    Randla-net: Efficient semantic segmentation of large-scale point clouds

    Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham. Randla-net: Efficient semantic segmentation of large-scale point clouds. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11108–1...

    2020

  24. [32]

    Scf-net: Learning spatial contextual features for large-scale point cloud segmentation

    Siqi Fan, Qiulei Dong, Fenghua Zhu, Yisheng Lv, Peijun Ye, and Fei-Yue Wang. Scf-net: Learning spatial contextual features for large-scale point cloud segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14504–14513, 2021

    2021

  25. [33]

    Rpvnet: A deep and efficient range-point-voxel fusion network for lidar point cloud segmentation

    Jianyun Xu, Ruixiang Zhang, Jian Dou, Yushi Zhu, Jie Sun, and Shiliang Pu. Rpvnet: A deep and efficient range-point-voxel fusion network for lidar point cloud segmentation. InProceedings of the IEEE/CVF international conference on computer vision, pages 16024–16033, 2021

    2021

  26. [34]

    Cylindrical and asymmetrical 3d convolution networks for lidar segmentation

    Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, and Dahua Lin. Cylindrical and asymmetrical 3d convolution networks for lidar segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9939–9948, 2021

    2021

  27. [35]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  28. [36]

    Pct: Point cloud transformer.Computational Visual Media, 7:187–199, 2021

    Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. Pct: Point cloud transformer.Computational Visual Media, 7:187–199, 2021

    2021

  29. [37]

    Learning to steer by mimicking features from heterogeneous auxiliary networks

    Yuenan Hou, Zheng Ma, Chunxiao Liu, and Chen Change Loy. Learning to steer by mimicking features from heterogeneous auxiliary networks. InProceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 8433–8440, 2019. 11

    2019

  30. [38]

    A gift from knowledge distillation: Fast optimization, network minimization and transfer learning

    Junho Yim, Donggyu Joo, Jihoon Bae, and Junmo Kim. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4133–4141, 2017

    2017

  31. [39]

    Towards efficient 3d object detection with knowledge distillation.Advances in Neural Information Processing Systems, 35:21300–21313, 2022

    Jihan Yang, Shaoshuai Shi, Runyu Ding, Zhe Wang, and Xiaojuan Qi. Towards efficient 3d object detection with knowledge distillation.Advances in Neural Information Processing Systems, 35:21300–21313, 2022

    2022

  32. [40]

    X3kd: Knowledge distillation across modalities, tasks and stages for multi- camera 3d object detection

    Marvin Klingner, Shubhankar Borse, Varun Ravi Kumar, Behnaz Rezaei, Venkatraman Narayanan, Senthil Yogamani, and Fatih Porikli. X3kd: Knowledge distillation across modalities, tasks and stages for multi- camera 3d object detection. InProceedings of the IEEE/CVF Conference on C...

    2023

  33. [41]

    Label-guided knowledge distillation for continual semantic segmentation on 2d images and 3d point clouds

    Ze Yang, Ruibo Li, Evan Ling, Chi Zhang, Yiming Wang, Dezhao Huang, Keng Teck Ma, Minhoe Hur, and Guosheng Lin. Label-guided knowledge distillation for continual semantic segmentation on 2d images and 3d point clouds. InProceedings of the IEEE/CVF International Conference on C...

    2023

  34. [42]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  35. [43]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. A Supplement...

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.