Pith. sign in

REVIEW 6 major objections 5 minor 46 references

Structure-Aware Automatic Channel Pruning by Searching with Graph Embedding

T0 review · 6 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that ranking pruned networks by cosine similarity of GCN embeddings picks configurations that keep accuracy, enabling automatic channel pruning with 85% FLOPs reduction.

desk verdict A fresh but unvalidated pruning pipeline whose reported results are undermined by test-set selection, missing ImageNet experiments, and an unproven similarity-ranking assumption. read the letter →

arxiv 2506.11469 v1 pith:B5BADEEQ submitted 2025-06-13 cs.AI

Zifan Liu , Yuan Cao , Yanwei Yu , Heng Qi , Jie Gui This is my paper · ORCID
classification cs.AI
keywords channelpruninggraphconvolutionalnetworksstructure-awareratesearchcontrastivelearningnetworkcompressioncosinesimilaritymodeldeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes SACP, a method that treats a convolutional neural network as a graph and uses a graph convolutional network (GCN) to embed the original network and many pruned variants of it into a vector space. The paper's central claim is that cosine similarity between a candidate pruned network's embedding and the original network's embedding predicts how well that candidate will retain accuracy after pruning. With this ranking, SACP searches a discretized space of layer-wise pruning ratios, lightly fine-tuning the top-ranked candidates and fully retraining the best few. On CIFAR-10 it reports removing roughly 82–85% of FLOPs and over 92% of parameters from VGG-16 and ResNet-18 while keeping accuracy close to or above the unpruned baseline, and it claims better compression than existing baselines.

What carries the argument

The load-bearing machinery is the pair of graph representation and GCN encoder, together with the cosine similarity score $s^{(j)} = \cos(z^{(j)}, z_{\text{orig}})$ from Eq. (10). Each candidate pruned network becomes a directed acyclic graph whose nodes are layers and whose binary node features mark which channels are retained; the GCN encodes this into an embedding, and the contrastive loss (Eq. 8) with L1-norm-pruned positives and random-pruned negatives shapes the embedding space. The search space is the discrete set of per-layer pruning rates $r_l \in \{0, 0.1, \ldots, 0.9\}$ constrained by a global pruning threshold $R(r) \ge \tau$. This machinery lets the paper screen millions of candidates with one forward pass through the GCN before any fine-tuning.

What would settle it

Rank-correlation test: sample, say, 100 candidate configurations from the search space, compute their cosine similarities to the original embedding, lightly fine-tune each, and compute the Spearman rank correlation between similarity and final validation accuracy; a correlation near zero would show the similarity signal does not carry the claimed information and the search reduces to random screening.

Watch

Extended reading notes

Core claim

The discovery the paper asserts is that the embedding of a network graph, learned by contrastive training on L1-norm pruned variants (positive) versus randomly pruned variants (negative), carries enough structural information that a simple cosine similarity to the original network's embedding ranks pruning configurations by expected quality. Section 3.5 states this directly: a higher similarity score suggests the pruned model retains more of the original network's important features and is expected to perform better after pruning. The GCN encoder is trained on graphs whose node features are binary masks of retained channels, with layer topology encoded as edges, including residual skip connections. After training, the search step encodes the original model and a large pool of candidate configurations, selects the top-m by cosine similarity, fine-tunes them, then fully retrains the top-k by validation accuracy to pick the final pruned network.

Load-bearing premise

The paper assumes, without measuring it, that a candidate pruned network's cosine similarity to the original network in the learned embedding space predicts how accurately that candidate will perform after fine-tuning.

Editorial extensions

If this is right

  • If cosine similarity in the learned embedding space truly tracks pruning quality, then a single forward pass through the GCN can rank millions of candidate pruning configurations without retraining each one.
  • The same graph encoder can be reused across different backbone architectures (VGG-16, ResNet-18, ResNet-56) with only the graph structure changed, so pruning becomes a search over binary masks rather than a hand-designed per-layer rule.
  • The global pruning threshold $\tau$ gives a practical control knob: a deployment team can set the minimum compression level and let the search find the best accuracy within that budget.
  • The two-stage selection (top-m light fine-tuning, then top-k full retraining) allows the search to rely on cheap signals for screening while reserving expensive training for the most promising candidates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One check the paper does not run is a direct correlation between similarity scores and post-fine-tuning accuracy; doing so on a random sample of candidates with the same global pruning threshold would show whether the ranking signal is real or whether the final accuracy comes mainly from the retraining stage.
  • Because the graph features are binary masks only, the encoder never sees weight magnitudes; if similarity ranking transfers across weight initializations or training epochs, the method could be used for early-exit or training-free pruning decisions, but the paper does not test that.
  • The contrastive supervision signal is implicitly tied to L1-norm pruning as the definition of 'good' pruning; replacing the positive-sample generator with another heuristic would shift what the embedding ranks as good, which is an untested design choice.
  • A natural extension is to apply the same graph-embedding search to layer-level pruning or to non-convolutional blocks (transformers), since the graph abstraction does not depend on convolution specifically; the paper leaves that unexplored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The paper proposes SACP, a channel-pruning framework that represents a CNN as a DAG with per-channel binary masks, trains a GCN encoder using a contrastive loss on L1-pruned and randomly pruned variants, and then searches a discretized pruning-rate space by ranking candidates by cosine similarity between the candidate embedding and the original network embedding. The top-ranked candidates are lightly fine-tuned, the top-k are fully retrained, and the final model is selected by test accuracy. The reported experiments claim strong compression with competitive accuracy on VGG-16, ResNet-18, and ResNet-56 on CIFAR-10, and the abstract additionally claims ImageNet experiments.

Significance. If the similarity-ranking mechanism in Eq. (10) were validated, the framework would offer an attractive way to automate channel-rate search with structural information. The paper is transparent about its search space and spells out Algorithm 1, and the graph encoding is a reasonable way to couple topology with pruning state. However, the empirical support for the central mechanism is missing: there is no validation that cosine similarity to the original embedding ranks candidates by quality, no random baseline, and the final selection on test labels undermines the reported numbers. The absence of the claimed ImageNet results and the internal numerical inconsistencies further reduce confidence in the experimental claims.

major comments (6)
  1. [Section 3.5, Eq. (10)] The central ranking mechanism is unvalidated: the paper never reports a correlation between the similarity score s(j) and actual post-pruning accuracy, never compares top-similarity with low-similarity candidates at matched compression, and never replaces the similarity filter with a random baseline in an ablation. Without one of these tests, the claim that higher cosine similarity to the original embedding predicts better pruning quality is unsupported, and the downstream results could be produced by the retraining and selection pipeline alone.
  2. [Section 4.2 / Algorithm 1] The final model is selected using test accuracy: Section 4.2 states 'Final performance is reported based on the best test accuracy' and Algorithm 1 returns the 'Final pruned model with best accuracy.' This makes the reported numbers best-of-k selections made on the test labels, so they are not unbiased estimates of pruning quality and do not support the claim of a fully automated, structure-aware search. The validation-based top-k selection should be followed by a single final evaluation on the test set, not selection by test accuracy.
  3. [Abstract / Section 4] The abstract and introduction state that experiments are conducted on both CIFAR-10 and ImageNet, but Section 4 contains no ImageNet results; all tables and ablations are on CIFAR-10. The abstract's claim of 'extensive experiments on benchmark datasets (CIFAR-10, ImageNet)' is therefore unsupported by the manuscript as written.
  4. [Table 3 / Section 4.3] The reported ResNet-56 result drops from 94.58% baseline accuracy to 90.06%, a 4.52-point degradation, while the text says SACP 'caused only a small accuracy degradation.' This drop, at 78% FLOP reduction, is not small relative to the baselines in Table 3 and needs to be discussed as a trade-off, ideally with comparisons at matched FLOPs.
  5. [Section 4.4 / Table 4] The text reports the full SACP ResNet-18 accuracy as 91.54%, while Table 4 reports 93.41% for the same configuration; Table 4 also displays the SACP-1 row as '88.7391.4991.32' with no column separators. These inconsistencies make the experimental results difficult to interpret and must be corrected before the results can be assessed.
  6. [Section 3.4] The positive training samples are generated using L1-norm pruning, so the contrastive encoder may be learning a representation of L1-style channel importance rather than global structural information. The paper provides no matched-compression comparison against L1 pruning or against a similarity ranking based on random embeddings, so the claimed structure-aware advantage is not demonstrated.
minor comments (5)
  1. [Abstract / Keywords] "KeywordsChannel" is missing a space; it should be "Keywords: Channel Pruning; ...".
  2. [Sections 1 and 3.4] The method is described as "unsupervised comparison learning" in Section 1 and as "supervised contrast learning" in Section 3.4; the terminology should be made consistent.
  3. [Table 3] The GAL-0.6 FLOPs entry "78.30" is missing units (M) and appears incomplete; the row formatting in the ResNet-56 block makes the columns difficult to read.
  4. [Section 4.2 and Algorithm 1] The "light fine-tuning" stage for the top-m candidates is not specified; reporting the number of epochs, learning rate, and data splitting for this stage would improve reproducibility.
  5. [References] Reference [32] is formatted inconsistently as "Cornell University - arXiv, Aug 2016" rather than as a conference paper; the reference list should be normalized.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported accuracy is selected by test-set oracle rather than predicted by the embedding-similarity ranking, so the central claim is only partially supported.

  1. fitted input called prediction [Section 3.5 and Section 4.2, Algorithm 1 line 16]
    "The final pruned model is chosen based on the test performance of these top-k configurations. ... Final performance is reported based on the best test accuracy. ... return Final pruned model with best accuracy"

    The reported Top-1 accuracy is, by the paper's own protocol, the maximum test accuracy among the fully retrained top-k candidates: the final model is selected using the test set, and the final performance is the best test accuracy. Test accuracy is therefore the selection objective, not an independent evaluation of the embedding-similarity ranking in Eq. (10). Any pool of k retrained candidates would produce a 'best test accuracy' even if s(j)=cos(z(j), z_orig) carried no signal about pruning quality. The reported gains therefore cannot validate the central claim that higher similarity predicts better pruned performance; the prediction reduces to oracle selection on the reported metric.

full rationale

The paper's derivation chain has one significant circular step: the final pruned model is chosen by test accuracy, and the same test accuracy is then reported as the method's result. This means the headline numbers are best-of-k selections on the evaluation metric, not predictions from the learned embedding similarity. The claimed indicator property of Eq. (10) is never independently tested against random ranking, low-similarity candidates, or a correlation analysis; the downstream oracle selection can produce strong reported numbers even if the similarity score carries no signal. There is no self-citation chain or imported uniqueness theorem, and the GCN training itself is not circular by construction, but the final evaluation protocol makes the central empirical claim partially circular. Score 6 reflects this partial circularity: the final model selection reduces by construction to choosing the best test accuracy, while the embedding component remains not directly validated.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The framework depends on several user-chosen thresholds and design choices: the global pruning threshold tau, the 0.1 discretization step, the contrastive temperature, the training and candidate counts, and the definition of positive samples via L1 pruning. None of these are derived from data or given a sensitivity analysis, and the embedding-similarity proxy is not validated. The axioms reflect the untested assumptions that GCN embeddings capture pruning quality and that cosine similarity to the original model predicts accuracy.

free parameters (5)
  • global pruning threshold tau = 0.6 (VGG-16), 0.7 (ResNet-18), 0.3 (ResNet-56)
    User-defined per model; controls the candidate search space and is not derived from data.
  • discretization step 0.1 = R = {0, 0.1, ..., 0.9}
    Arbitrary granularity; no sensitivity analysis is provided.
  • contrastive temperature tau in Eq. 5 = not reported
    A key hyperparameter for the contrastive loss, but its value is omitted.
  • candidate counts (N, pool size, m, k) = e.g., 10k training, 1M candidates, m=150, k=10 for ResNet-18
    Tuned per model; only k and m receive a limited sensitivity analysis on ResNet-18.
  • positive/negative sampling scheme = L1-norm positives, random negatives
    The definition of good and bad pruning is assumed without justification; no alternatives are tested.
assumptions (4)
  • domain assumption GCN embeddings of binary channel-mask graphs capture architectural quality relevant to pruning.
    Relied on throughout Section 3.4; no validation that embedding similarity correlates with accuracy.
  • ad hoc to paper Cosine similarity between candidate and original model embeddings is a reliable proxy for post-pruning performance.
    Stated in Section 3.5, Eq. 10; unverified and contradicted by ResNet-56 results where high-similarity candidates still lose 4.5% accuracy.
  • domain assumption L1-norm pruning preserves high-frequency channels and maintains reasonable performance.
    Used to generate positive samples in Section 3.4; drawn from prior work [32], but no evidence here that it is the best positive definition.
  • ad hoc to paper A pruning rate discretization of 0.1 is fine enough to find near-optimal configurations.
    No analysis of granularity effect; Section 3.3 defines R without justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Structure-Aware Automatic Channel Pruning by Searching with Graph Embedding." pith.science (2026). https://pith.science/paper/B5BADEEQ

@misc{pith2026250611469,
  author       = {Pith},
  title        = {Pith review of: Structure-Aware Automatic Channel Pruning by Searching with Graph Embedding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B5BADEEQ}},
  note         = {Machine review of arXiv:2506.11469}
}
read the original abstract

Channel pruning is a powerful technique to reduce the computational overhead of deep neural networks, enabling efficient deployment on resource-constrained devices. However, existing pruning methods often rely on local heuristics or weight-based criteria that fail to capture global structural dependencies within the network, leading to suboptimal pruning decisions and degraded model performance. To address these limitations, we propose a novel structure-aware automatic channel pruning (SACP) framework that utilizes graph convolutional networks (GCNs) to model the network topology and learn the global importance of each channel. By encoding structural relationships within the network, our approach implements topology-aware pruning and this pruning is fully automated, reducing the need for human intervention. We restrict the pruning rate combinations to a specific space, where the number of combinations can be dynamically adjusted, and use a search-based approach to determine the optimal pruning rate combinations. Extensive experiments on benchmark datasets (CIFAR-10, ImageNet) with various models (ResNet, VGG16) demonstrate that SACP outperforms state-of-the-art pruning methods on compression efficiency and competitive on accuracy retention.

Figures

Figures reproduced from arXiv: 2506.11469 by the authors.

Figure 1
Figure 1. The overall framework of the proposed SACP method. In the training phase (upper part), a set of pruned [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Effect of k and m values on pruning performance with ResNet-18 on CIFAR-10. As shown in [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 34 canonical work pages

  1. [1]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  2. [2]

    Spiking-yolo: spiking neural network for energy-efficient object detection

    Seijoon Kim, Seongsik Park, Byunggook Na, and Sungroh Yoon. Spiking-yolo: spiking neural network for energy-efficient object detection. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 11270–11277, 2020

    work page 2020

  3. [3]

    Scribblesup: Scribble-supervised convolutional networks for semantic segmentation

    Di Lin, Jifeng Dai, Jiaya Jia, Kaiming He, and Jian Sun. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3159–3167, 2016

    work page 2016

  4. [4]

    Structadmm: Achieving ultrahigh efficiency in structured pruning for dnns.IEEE transactions on neural networks and learning systems, 33(5):2259–2273, 2021

    Tianyun Zhang, Shaokai Ye, Xiaoyu Feng, Xiaolong Ma, Kaiqi Zhang, Zhengang Li, Jian Tang, Sijia Liu, Xue Lin, Yongpan Liu, et al. Structadmm: Achieving ultrahigh efficiency in structured pruning for dnns.IEEE transactions on neural networks and learning systems, 33(5):2259–2273, 2021. 10 Running Title for Header

    work page 2021

  5. [5]

    Convolutional neural network pruning with structural redundancy reduction

    Zi Wang, Chengcheng Li, and Xiangyang Wang. Convolutional neural network pruning with structural redundancy reduction. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 14913–14922, 2021

    work page 2021

  6. [6]

    Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks

    Pengtao Xu, Jian Cao, Fanhua Shang, Wenyu Sun, and Pu Li. Layer pruning via fusible residual convolutional block for deep neural networks.ArXiv preprint arXiv:2011.14356, 2020

    work page Pith review arXiv 2011

  7. [7]

    Autocompress: An automatic dnn structured pruning framework for ultra-high compression rates

    Ning Liu, Xiaolong Ma, Zhiyuan Xu, Yanzhi Wang, Jian Tang, and Jieping Ye. Autocompress: An automatic dnn structured pruning framework for ultra-high compression rates. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 4876–4883, 2020

    work page 2020

  8. [8]

    Hawq: Hessian aware quantization of neural networks with mixed-precision

    Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Hawq: Hessian aware quantization of neural networks with mixed-precision. InProceedings of the IEEE international conference on computer vision, pages 293–302, 2019

    work page 2019

Show all 46 references
  1. [9]

    Emq: Evolving training-free proxies for automated mixed precision quantization

    Peijie Dong, Lujun Li, Zimian Wei, Xin Niu, Zhiliang Tian, and Hengyue Pan. Emq: Evolving training-free proxies for automated mixed precision quantization. InProceedings of the IEEE international conference on computer vision, pages 17076–17086, 2023

    2023

  2. [10]

    Towards unified int8 training for convolutional neural network

    Feng Zhu, Ruihao Gong, Fengwei Yu, Xianglong Liu, Yanfei Wang, Zhelong Li, Xiuqi Yang, and Junjie Yan. Towards unified int8 training for convolutional neural network. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1969–1979, 2020

    1969

  3. [11]

    Haq: Hardware-aware automated quantization with mixed precision

    Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han. Haq: Hardware-aware automated quantization with mixed precision. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 8612–8620, 2019

    2019

  4. [12]

    Teachers do more than teach: Compressing image-to-image models

    Qing Jin, Jian Ren, Oliver J Woodford, Jiazhuo Wang, Geng Yuan, Yanzhi Wang, and Sergey Tulyakov. Teachers do more than teach: Compressing image-to-image models. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 13600–13611, 2021

    2021

  5. [13]

    Relational knowledge distillation

    Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3967–3976, 2019

    2019

  6. [14]

    Knowledge distillation: A survey.Interna- tional journal of computer vision, 129(6):1789–1819, 2021

    Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey.Interna- tional journal of computer vision, 129(6):1789–1819, 2021

    2021

  7. [15]

    Channel pruning for accelerating very deep neural networks

    Yihui He, Xiangyu Zhang, and Jian Sun. Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE international conference on computer vision, pages 1389–1397, 2017

    2017

  8. [16]

    Thinet: A filter level pruning method for deep neural network compression

    Jian-Hao Luo, Jianxin Wu, and Weiyao Lin. Thinet: A filter level pruning method for deep neural network compression. InProceedings of the IEEE international conference on computer vision, pages 5058–5066, 2017

    2017

  9. [17]

    Channel pruning via automatic structure search.ArXiv preprint arXiv:2001.08565, 2020

    Mingbao Lin, Rongrong Ji, Yuxin Zhang, Baochang Zhang, Yongjian Wu, and Yonghong Tian. Channel pruning via automatic structure search.ArXiv preprint arXiv:2001.08565, 2020

    2001 arXiv

  10. [18]

    Amc: Automl for model compression and acceleration on mobile devices

    Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. Amc: Automl for model compression and acceleration on mobile devices. InProceedings of the European conference on computer vision, pages 784–800, 2018

    2018

  11. [19]

    Filter pruning via automatic pruning rate search

    Qiming Sun, Shan Cao, and Zhixiang Chen. Filter pruning via automatic pruning rate search. InProceedings of the Asian conference on computer vision, pages 4293–4309, 2022

    2022

  12. [20]

    Shufflenet: An extremely efficient convolutional neural network for mobile devices

    Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 6848–6856, 2018

    2018

  13. [21]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications

    Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. ArXiv preprint arXiv:1704.04861, 2017

    2017 arXiv

  14. [22]

    Mobilenetv2: Inverted residuals and linear bottlenecks

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018

    2018

  15. [23]

    Q-bert: Hessian based ultra low precision quantization of bert

    Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Q-bert: Hessian based ultra low precision quantization of bert. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 8815–8821, 2020

    2020

  16. [24]

    Post-training quantization for vision transformer.Advances in neural information processing systems, 34:28092–28103, 2021

    Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. Post-training quantization for vision transformer.Advances in neural information processing systems, 34:28092–28103, 2021. 11 Running Title for Header

    2021

  17. [25]

    Zeroq: A novel zero shot quantization framework

    Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Zeroq: A novel zero shot quantization framework. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13169–13178, 2020

    2020

  18. [26]

    Distilling the knowledge in a neural network.ArXiv preprint arXiv:1503.02531, 2015

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.ArXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  19. [27]

    Contrastive distillation on intermediate representations for language model compression.ArXiv preprint arXiv:2009.14167, 2020

    Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang, and Jingjing Liu. Contrastive distillation on intermediate representations for language model compression.ArXiv preprint arXiv:2009.14167, 2020

    2009 arXiv

  20. [28]

    Class attention transfer based knowledge distillation

    Ziyao Guo, Haonan Yan, Hui Li, and Xiaodong Lin. Class attention transfer based knowledge distillation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 11868–11877, 2023

    2023

  21. [29]

    Exploiting linear structure within convolutional networks for efficient evaluation.Advances in neural information processing systems, 27, 2014

    Emily L Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus. Exploiting linear structure within convolutional networks for efficient evaluation.Advances in neural information processing systems, 27, 2014

    2014

  22. [30]

    Speeding up convolutional neural networks with low rank expansions.ArXiv preprint arXiv:1405.3866, 2014

    Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. Speeding up convolutional neural networks with low rank expansions.ArXiv preprint arXiv:1405.3866, 2014

    2014 arXiv

  23. [31]

    Low-rank matrix factorization for deep neural network training with high-dimensional output targets

    Tara N Sainath, Brian Kingsbury, Vikas Sindhwani, Ebru Arisoy, and Bhuvana Ramabhadran. Low-rank matrix factorization for deep neural network training with high-dimensional output targets. InIEEE international conference on acoustics, speech and signal processing, pages 6655–6...

    2013

  24. [32]

    Pruning filters for efficient convnets

    Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and HansPeter Graf. Pruning filters for efficient convnets. Cornell University - arXiv, Aug 2016

    2016

  25. [33]

    Hrank: Filter pruning using high-rank feature map

    Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, and Ling Shao. Hrank: Filter pruning using high-rank feature map. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1529–1538, 2020

    2020

  26. [34]

    Learning efficient convolutional networks through network slimming

    Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. Learning efficient convolutional networks through network slimming. InProceedings of the IEEE international conference on computer vision, pages 2736–2744, 2017

    2017

  27. [35]

    Metaprun- ing: Meta learning for automatic neural network channel pruning

    Zechun Liu, Haoyuan Mu, Xiangyu Zhang, Zichao Guo, Xin Yang, Kwang-Ting Cheng, and Jian Sun. Metaprun- ing: Meta learning for automatic neural network channel pruning. InProceedings of the IEEE international conference on computer vision, pages 3296–3305, 2019

    2019

  28. [36]

    Auto graph encoder-decoder for neural network pruning

    Sixing Yu, Arya Mazaheri, and Ali Jannesari. Auto graph encoder-decoder for neural network pruning. In Proceedings of the IEEE international conference on computer vision, pages 6362–6372, 2021

    2021

  29. [37]

    Automatic network pruning via hilbert-schmidt independence criterion lasso under information bottleneck principle

    Song Guo, Lei Zhang, Xiawu Zheng, Yan Wang, Yuchao Li, Fei Chao, Chenglin Wu, Shengchuan Zhang, and Rongrong Ji. Automatic network pruning via hilbert-schmidt independence criterion lasso under information bottleneck principle. InProceedings of the IEEE international conferenc...

    2023

  30. [38]

    Semi-supervised classification with graph convolutional networks.ArXiv preprint arXiv:1609.02907, 2016

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks.ArXiv preprint arXiv:1609.02907, 2016

    2016 arXiv

  31. [39]

    Graph attention networks.ArXiv preprint arXiv:1710.10903, 2017

    Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks.ArXiv preprint arXiv:1710.10903, 2017

    2017 arXiv

  32. [40]

    Graph structure of neural networks

    Jiaxuan You, Jure Leskovec, Kaiming He, and Saining Xie. Graph structure of neural networks. InInternational conference on machine learning, pages 10881–10891. PMLR, 2020

    2020

  33. [41]

    Proxylessnas: Direct neural architecture search on target task and hardware

    Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search on target task and hardware. ArXiv preprint arXiv:1812.00332, 2018

    2018 arXiv

  34. [42]

    Progressive neural architecture search

    Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. InProceedings of the European conference on computer vision, pages 19–34, 2018

    2018

  35. [43]

    Automl: A survey of the state-of-the-art.Knowledge-based systems, 212:106622, 2021

    Xin He, Kaiyong Zhao, and Xiaowen Chu. Automl: A survey of the state-of-the-art.Knowledge-based systems, 212:106622, 2021

    2021

  36. [44]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. InInternational conference on machine learning, pages 1597–1607, 2020

    2020

  37. [45]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 9729–9738, 2020

    2020

  38. [46]

    Inductive representation learning on large graphs.Advances in neural information processing systems, 30, 2017

    Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs.Advances in neural information processing systems, 30, 2017. 12

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.