Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Partial Channel Network: Compute Fewer, Perform Better

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single module that splits feature channels into a convolution branch and an attention branch can replace both regular convolution and regular visual attention while cutting parameters and FLOPs.

desk verdict Useful partial-conv-plus-attention module with transferable gains, but headline accuracy claims rest on mixed recipes and a wrong complexity formula. read the letter →

arxiv 2502.01303 v1 pith:553WQC62 submitted 2025-02-03 cs.CV cs.AI

Haiduo Huang , Tian Xia , Wenzhe zhao , Pengju Ren This is my paper · ORCID
classification cs.CVcs.AI
keywords partialchannelmechanismattentionconvolutionefficientneuralnetworksvisualImageNet-1KclassificationCOCOdetectionandsegmentationdynamichybridvisionbackbone
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a partial channel mechanism (PCM): split a feature map into channel groups and run a different operation on each group in parallel. Its main module, PATConv, sends one fraction of channels through a 3x3 convolution and the other fraction through a visual attention block, so attention is computed only on a subset of channels. The claim is that this single module can replace both regular convolution and regular visual attention, reducing parameters and FLOPs while improving or matching accuracy. Building on it, PartialNet gives a family of hybrid backbones that the authors report outperform efficient baselines on ImageNet-1K and COCO. If correct, the result means a cheap channel-split design, not full attention, is enough to capture the global information that attention is supposed to provide.

What carries the argument

The central object is the partial channel mechanism: a split operation divides the input feature map into two channel groups, each handled by a different parallel operator, followed by concatenation. The specific engine is PATConv, whose novelty is running convolution and attention side by side on disjoint channel subsets rather than serially. Three instantiations carry the accuracy gains: PAT_ch pairs a 3x3 convolution with an enhanced Gaussian channel-attention (mean and standard deviation squeeze), PAT_sp pairs a 1x1 convolution with a spatial-attention map and can be folded into adjacent MLP layers, and PAT_sf pairs a 3x3 convolution with self-attention with relative position encoding in the last stage. DPConv supplies the learnable split ratio through a Kronecker product of 2x2 binary gates trained with a straight-through estimator.

What would settle it

Train PartialNet-T2 twice under the paper's ImageNet-1K settings: once with the attention branch inside PAT_ch and once with that branch replaced by a second 3x3 convolution of matched parameter cost. If top-1 accuracy stays within the noise level, the reported gains come from the extra parameters and channel split, not from partial attention; if accuracy drops clearly, partial attention is doing the work.

Watch

Extended reading notes

Core claim

The paper's central claim is that a single module, PATConv, can completely replace both a regular convolution and a regular visual attention layer while using fewer parameters and FLOPs. Formally, with input $F \in \mathbb{R}^{h \times w \times c_{\text{in}}}$ and output $O$, PATConv computes $O = \mathrm{Conv}(F_{c_{\text{in}} r_p}) \cup \mathrm{Atten}(F_{c_{\text{in}}(1-r_p)})$: a convolution branch processes the fraction $r_p$ of channels and an attention branch processes the rest in parallel, then the two outputs are concatenated. The paper develops three attention variants for that branch—channel attention (PAT_ch), spatial attention (PAT_sp), and self-attention (PAT_sf)—and a learnable dynamic partial convolution that adapts $r_p$ layer by layer. The reported experiments show the resulting PartialNet family beating FasterNet at lower FLOPs on ImageNet-1K and improving COCO detection and segmentation over FasterNet backbones.

Load-bearing premise

The load-bearing premise is that attention applied to only a fraction of the channels captures most of the benefit of full attention, so the remaining channels do not need their own attention.

Editorial extensions

If this is right

  • A network built from PATConv no longer needs a separate attention module: the three PAT blocks cover channel, spatial, and self-attention, so the same module set serves both local and global interaction.
  • Because PATConv is a convolution replacement, it can be inserted into existing CNN backbones; the paper's transfer experiments report accuracy gains on ResNet50, MobileNetV2, and ConvNeXt-tiny with higher throughput.
  • The adaptive DPConv ratio means the split between cheap convolution and expensive attention can be tuned per layer to meet a specified complexity budget, avoiding manual search over $r_p$.
  • At matched model sizes, the reported ImageNet-1K numbers imply the accuracy-FLOPs frontier is shifted: PartialNet-T2 reaches 80.2 percent top-1 at 1.03 G FLOPs, above FasterNet-T2's 78.9 percent at 1.91 G FLOPs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves the attention-versus-split attribution open; a diagnostic that would separate the two effects is to replace the attention branch with a second convolution of the same cost and compare top-1 accuracy.
  • The learned split-ratio pattern—first and last layers stay denser while middle layers become sparse—mirrors results from quantization studies; the same ordering could be tested as a prior for pruning or mixed-precision allocation.
  • Since PATConv is defined for any attention module, the three PAT blocks are only a sample of the recipe; applying the identical split-and-run-in-parallel pattern to other modalities, such as video or point cloud backbones, is a direct transfer test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a partial channel mechanism (PCM) that splits feature-map channels and applies different operations to each part. On this basis it introduces Partial Attention Convolution (PATConv), three concrete blocks (PAT_ch, PAT_sp, PAT_sf), a Dynamic Partial Convolution (DPConv) intended to learn split ratios, and a PartialNet family. The central claims are that PATConv can replace both regular convolution and regular visual attention, and that PartialNet achieves superior top-1 accuracy and inference speed compared with state-of-the-art efficient models on ImageNet-1K, with strong transfer to COCO detection and segmentation. Experiments include classification tables, COCO results, ablations on attention type and convolution type, and a supplementary same-recipe reproduction of FasterNet.

Significance. If the main claims held, the partial-channel combination of convolution and attention would be a practically valuable efficient operator, because it offers a simple, hardware-friendly way to reduce FLOPs and parameters while preserving accuracy. The paper has real strengths: it provides a code link, extensive ImageNet and COCO results, a same-recipe reproduction of FasterNet in the supplement (Tab. 13), and ablation studies that isolate the contribution of each PAT block. However, the current evidence does not establish the headline 'superior accuracy' claim at the larger model scales, and the DPConv derivation contains demonstrable errors. The proposed mechanism may still be a useful engineering contribution, but the manuscript needs substantial correction and re-benchmarking before the central claims are supported.

major comments (4)
  1. [Supplementary §7, Tab. 13 vs Tab. 1] Table 1's headline comparison mixes baselines trained under different recipes, as the supplement itself concedes ('For other models without ADT, such as ShuffleNetV2, MobileNetV2, and GhostNet, although the comparison is not entirely fair, we include them for reference'). Table 13 provides a same-recipe reproduction of FasterNet; comparing against those reproduced numbers, the PartialNet margins become 2.9/1.6/1.0/0.6/0.1/0.3 points for T0/T1/T2/S/M/L. At M and L the advantage is only 0.1 and 0.3 points, which is within typical ImageNet run-to-run variance. The abstract's claim of 'superior top-1 accuracy' is therefore not established at the larger scales. Please provide same-recipe numbers for all compared backbones, or restrict the claim to the scales where the margin is robust.
  2. [§3.2, Eq. (7)] The paper states that Eq. (7) counts the number of non-zero elements in the mask U of the l-th DPConv layer. But the expression (row-sum) times (column-sum) is not that count in general. For example, with K=2 and g=(0,0), U is the 4x4 identity matrix, which has four non-zero entries, whereas the formula gives 1. The regularization term ζ and the Pareto objective in Eq. (9) therefore do not measure the stated computational complexity. Please rederive Eq. (7) and the associated κ/ζ constraint.
  3. [§3.2, Eq. (5)] The mask definition in Eq. (5) is not well-formed. Since r_p is a ratio, 1/r_p is not a channel index; as written, the condition i < 1/r_p would select only the first three channels when r_p=1/4, independent of c_in. The connection between the Kronecker-gate representation and the actual channel mask needs a clean formal statement before the adaptive split ratio can be evaluated.
  4. [§3.2 vs Tab. 9] The abstract and Section 3.2 present DPConv as a mechanism that 'adaptively learn[s] the proportion of split channels', but Tab. 9 fixes r_p=1/4 for every PAT block in every PartialNet variant, and no reported model in Tabs. 1-2 is trained with learned ratios. Fig. 7 shows learned ratios only for T0 and does not report top-1 accuracy for those configurations. Moreover, the constraint target κ uses θ=4, explicitly chosen to match FasterNet's r_p=1/4 complexity, so the regularizer is enforcing a hand-picked target rather than deriving the split from an independent principle. Either train and report final models with learned r_p, or present DPConv as a separate analysis tool rather than as a component of the evaluated models.
minor comments (6)
  1. [§3.1, Eq. (1)] The notation F^{c_in × r_p} is not a standard way to denote a channel subset; please use a mask or channel-range notation so the split is unambiguous.
  2. [Fig. 3 and Fig. 4] The symbol U denotes both the binary mask matrix in DPConv and the concatenation operation in the PAT block figures; please rename one of them to avoid confusion.
  3. [Supplementary Tab. 13] In the FasterNet-S rows, one row appears to be a reproduced result but does not carry the '*' marker; please clarify which entries are original and which are reproduced under the authors' setup.
  4. [§4.1 and Supplementary Tab. 8] The text says the authors use 'the same regularization and augmentation techniques as FasterNet', but Tab. 8 shows different mixup, cutmix, and stochastic-depth values across variants; please state that the training recipe is of the same family rather than identical.
  5. [§3.2, Eq. (8)] The summation index i in Eq. (8) is inconsistent with the gate subscript k used in the rest of the equation; please align the notation.
  6. [Fig. 7] The θ sweep reports only learned ratios and not the resulting top-1 accuracies; including accuracy values would let the reader judge whether the adapted ratios actually improve the trade-off.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central PATConv/PartialNet claims rest on ablations and external baselines, not on self-referential fits.

full rationale

The derivation chain is empirical rather than definitional. Eq. (1) defines PATConv as Conv on rp channels concatenated with attention on the remaining channels; the claim that attention need only be applied to a subset is tested directly in Tab. 3 (partial vs. full attention) and Tab. 4 (progressive block ablations), so the conclusion is not contained in the definition. The DPConv section (Eqs. 2-9) is a constrained optimization: zeta measures the number of activated weights, kappa is a user-set target complexity, and the loss penalizes zeta > kappa. Learned ratios in Fig. 7 are therefore outputs of a regularized search anchored to the chosen theta, not predictions fitted to the reported accuracies; this weakens the novelty of the adaptive claim but does not make the main ImageNet results circular. Two genuine concerns lie outside circularity: Tab. 9 fixes rp = 1/4 in the final PartialNet variants, so DPConv is not load-bearing for the headline numbers; and Tab. 1 mixes heterogeneous training recipes, although the supplementary (Sec. 7, Tab. 13) provides same-recipe FasterNet reproductions and shows reduced but nonzero margins at M/L scale. Eq. (7) also appears to misstate the total number of ones in U (row and column sums are multiplied instead of multiplying by the number of rows), which is an algebraic correctness issue, not a circularity. No self-citation chains or imported uniqueness theorems are present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The ledger shows that the central accuracy-efficiency claims do not depend on invented physical entities. The main input costs are the split ratio hyperparameter, the regularization constants, and the domain assumption that attention on partial channels retains global information. No new particles or mediators are introduced.

free parameters (4)
  • split ratio rp = 1/4 in architecture (Tab 9); learned in DPConv Sec 3.2
    Controls the proportion of channels assigned to convolution vs attention; treated as a hyperparameter in Eq (1), fixed in all reported final models.
  • regularization penalty exponent alpha = -0.01 when zeta > kappa, else 0
    Empirically set constant in the weighted Pareto-optimal product in Sec 3.2.
  • penalty factor beta = 0.9
    Default penalty factor for the gate-ordering regularizer psi in Eq (9).
  • complexity constraint theta = varied in Fig 7; theta=4 corresponds to rp=1/4
    Sets the target complexity kappa in the DPConv regularizer; is a user-chosen constraint, not derived.
assumptions (4)
  • domain assumption Feature map channels are redundant, so computing on a subset retains most of the useful signal.
    Invoked in Sec 3.1 citing GhostNet and FasterNet; the basis for partial-channel processing.
  • domain assumption Feature maps approximately follow a normal distribution, so mean and variance are sufficient channel statistics.
    Sec 3.1, PAT_ch, citing BatchNorm's assumption; this justifies modifying SENet to include std.
  • domain assumption Running convolution and attention on separate channel groups in parallel improves GPU utilization.
    Sec 3.1, citing a parallel programming textbook; the speed claim rests on hardware behavior rather than a mathematical guarantee.
  • ad hoc to paper A split ratio mask U can be decomposed into Kronecker products of 2x2 blocks with the Sign function trained via straight-through estimator.
    Sec 3.2, Eqs 2-3; this modeling choice is specific to DPConv and constrains channel counts to powers of two.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Partial Channel Network: Compute Fewer, Perform Better." pith.science (2026). https://pith.science/paper/553WQC62

@misc{pith2026250201303,
  author       = {Pith},
  title        = {Pith review of: Partial Channel Network: Compute Fewer, Perform Better},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/553WQC62}},
  note         = {Machine review of arXiv:2502.01303}
}
read the original abstract

Designing a module or mechanism that enables a network to maintain low parameters and FLOPs without sacrificing accuracy and throughput remains a challenge. To address this challenge and exploit the redundancy within feature map channels, we propose a new solution: partial channel mechanism (PCM). Specifically, through the split operation, the feature map channels are divided into different parts, with each part corresponding to different operations, such as convolution, attention, pooling, and identity mapping. Based on this assumption, we introduce a novel partial attention convolution (PATConv) that can efficiently combine convolution with visual attention. Our exploration indicates that the PATConv can completely replace both the regular convolution and the regular visual attention while reducing model parameters and FLOPs. Moreover, PATConv can derive three new types of blocks: Partial Channel-Attention block (PAT_ch), Partial Spatial-Attention block (PAT_sp), and Partial Self-Attention block (PAT_sf). In addition, we propose a novel dynamic partial convolution (DPConv) that can adaptively learn the proportion of split channels in different layers to achieve better trade-offs. Building on PATConv and DPConv, we propose a new hybrid network family, named PartialNet, which achieves superior top-1 accuracy and inference speed compared to some SOTA models on ImageNet-1K classification and excels in both detection and segmentation on the COCO dataset. Our code is available at https://github.com/haiduo/PartialNet.

Figures

Figures reproduced from arXiv: 2502.01303 by the authors.

Figure 1
Figure 1. Comparison of different operation types. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our PartialNet achieves higher trade-off of accuracy and [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The overall architecture of our PartialNet, consisting of four hierarchical stages, each incorporating a series of PartialNet blocks [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The generation process of DPConv, where ⊙ denotes elementwise product, ⊛ denotes a Kronecker product. Considering the constraints of model deployment, e.g., the parameters and FLOPs, it is necessary to limit the learned rp to avoid being too large. Therefore, we design…
Figure 5
Figure 5. Figure 5: The relationship between FLOPs and Top-1 Accuracy in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Visualization results show different categories of the [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The learned split ratios rp of different layers of PartialNet-T0 under different complexity constraints θ. 5. Conclusion Feature selection theory shows that there may be a certain degree of redundancy and correlation between features. While this redundancy does not pro…
Figure 8
Figure 8. Figure 8: Detailed of three PATConv blocks. Where ⊙ and ⊗ denote element-wise multiplication and matrix multiplication respectively, and C = Cp + Cp ′ . (b) ConvNext-tiny PATConv (a) ConvNext-tiny original [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: The training process of ConvNext-tiny with and without PATConv (i.e., PAT [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FSDC-DETR: A Frequency-Spatial Domain Collaborative DETR for Small Object Detection

    cs.CV 2026-07 conditional novelty 5.5 of 10

    FSDC-DETR improves small object detection by explicitly modeling frequency-spatial representations through dual-branch adaptive fusion, shunt feature fusion, and wavelet-based dynamic downsampling, achieving state-of-...

Reference graph

Works this paper leans on

43 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    The kronecker product

    Bobbi Jo Broxson. The kronecker product. 2006. 4

    work page 2006

  2. [2]

    Efficientvit: Lightweight multi-scale attention for high- resolution dense prediction

    Han Cai, Junyan Li, Muyan Hu, Chuang Gan, and Song Han. Efficientvit: Lightweight multi-scale attention for high- resolution dense prediction. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17302– 17313, 2023. 3, 6, 4

    work page 2023

  3. [3]

    Run, don’t walk: Chasing higher flops for faster neural networks

    Jierun Chen, Shiu-hong Kao, Hao He, Weipeng Zhuo, Song Wen, Chul-Ho Lee, and S-H Gary Chan. Run, don’t walk: Chasing higher flops for faster neural networks. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12021–12031, 2023. 1, 2, 3, 6, 7, 4, 5

    work page 2023

  4. [4]

    Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1

    Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830 ,

    arXiv

  5. [5]

    Scaling up your kernels to 31x31: Revisiting large kernel design in cnns

    Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding. Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 11963–11975, 2022. 1, 2

    work page 2022

  6. [6]

    Hawq: Hessian aware quantization of neural networks with mixed-precision

    Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Ma- honey, and Kurt Keutzer. Hawq: Hessian aware quantization of neural networks with mixed-precision. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 293–302, 2019. 8

    work page 2019

  7. [7]

    Single- and multi-gpu computing on nvidia- and amd-based server platforms for solidification modeling application

    Kamil Halbiniak, Norbert Meyer, and Krzysztof Rojek. Single- and multi-gpu computing on nvidia- and amd-based server platforms for solidification modeling application. Concurrency and Computation: Practice and Experience, 36 (9):e8000, 2024. 6

    work page 2024

  8. [8]

    Ghostnet: More features from cheap operations

    Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1580–1589,

    work page

Show all 43 references
  1. [9]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015. 6, 7, 4, 5

    2015 arXiv

  2. [10]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 6

    2017

  3. [11]

    Conv2former: A simple transformer-style convnet for visual recognition

    Qibin Hou, Cheng-Ze Lu, Ming-Ming Cheng, and Jiashi Feng. Conv2former: A simple transformer-style convnet for visual recognition. arXiv preprint arXiv:2211.11943, 2022. 1

    2022 arXiv

  4. [12]

    Le, and Hartwig Adam

    Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V . Le, and Hartwig Adam. Searching for mobilenetv3. Proceedings of the IEEE/CVF international conference on computer vision , abs/1905....

    1905 arXiv

  5. [13]

    Mobilenets: Efficient convolu- tional neural networks for mobile vision applications

    Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco An- dreetto, and Hartwig Adam. Mobilenets: Efficient convolu- tional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017. 1

    2017 arXiv

  6. [14]

    Squeeze-and-excitation net- works

    Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation net- works. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 7132–7141, 2018. 2, 3, 5

    2018

  7. [15]

    Batch normalization: Accelerating deep network training by reducing internal co- variate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. In International conference on machine learn- ing, pages 448–456. pmlr, 2015. 3

    2015

  8. [16]

    Programming mas- sively parallel processors: a hands-on approach

    David B Kirk and W Hwu Wen-Mei. Programming mas- sively parallel processors: a hands-on approach . Morgan kaufmann, 2016. 3

    2016

  9. [17]

    Srm: A style-based recalibration module for convolutional neural networks

    HyunJae Lee, Hyo-Eun Kim, and Hyeonseob Nam. Srm: A style-based recalibration module for convolutional neural networks. In Proceedings of the IEEE/CVF International conference on computer vision, pages 1854–1862, 2019. 3, 5

    2019

  10. [18]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 6, 4

    2021

  11. [19]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 11976–11986,

  12. [20]

    Shufflenet v2: Practical guidelines for efficient cnn architec- ture design

    Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelines for efficient cnn architec- ture design. In Proceedings of the European conference on computer vision (ECCV), pages 116–131, 2018. 1, 2, 3, 6, 4

    2018

  13. [21]

    Mobilevit: light- weight, general-purpose, and mobile-friendly vision trans- former

    Sachin Mehta and Mohammad Rastegari. Mobilevit: light- weight, general-purpose, and mobile-friendly vision trans- former. arXiv preprint arXiv:2110.02178, 2021. 2, 7, 4, 5

    2021 arXiv

  14. [22]

    Separable self- attention for mobile vision transformers

    Sachin Mehta and Mohammad Rastegari. Separable self- attention for mobile vision transformers. arXiv preprint arXiv:2206.02680, 2022. 3, 6, 4

    2022 arXiv

  15. [23]

    Edgevits: Competing light-weight cnns on mobile devices with vision transformers

    Junting Pan, Adrian Bulat, Fuwen Tan, Xiatian Zhu, Lukasz Dudziak, Hongsheng Li, Georgios Tzimiropoulos, and Brais Martinez. Edgevits: Competing light-weight cnns on mobile devices with vision transformers. In European Conference on Computer Vision, pages 294–311. Springer, 2022. 2

    2022

  16. [24]

    Vision transformers are robust learners

    Sayak Paul and Pin-Yu Chen. Vision transformers are robust learners. In Proceedings of the AAAI conference on Artificial Intelligence, pages 2071–2081, 2022. 2

    2022

  17. [25]

    Do vision trans- formers see like convolutional neural networks? Advances in Neural Information Processing Systems, 34:12116–12128,

    Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision trans- formers see like convolutional neural networks? Advances in Neural Information Processing Systems, 34:12116–12128,

  18. [26]

    Hornet: Efficient high- order spatial interactions with recursive gated convolutions

    Yongming Rao, Wenliang Zhao, Yansong Tang, Jie Zhou, Ser Nam Lim, and Jiwen Lu. Hornet: Efficient high- order spatial interactions with recursive gated convolutions. 9 Advances in Neural Information Processing Systems , 35: 10353–10366, 2022. 1, 3

    2022

  19. [27]

    Mobilenetv2: Inverted residuals and linear bottlenecks

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zh- moginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 4510–4520, 2018. 1, 2, 6, 4

    2018

  20. [28]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE in- ternational conference on computer vision , pages 618–626,

  21. [29]

    Swiftformer: Efficient additive attention for transformer- based real-time mobile vision applications

    Abdelrahman Shaker, Muhammad Maaz, Hanoona Rasheed, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan. Swiftformer: Efficient additive attention for transformer- based real-time mobile vision applications. arXiv preprint arXiv:2303.15446, 2023. 3

    2023 arXiv

  22. [30]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105–6114. PMLR,

  23. [31]

    Efficientnetv2: Smaller models and faster training

    Mingxing Tan and Quoc Le. Efficientnetv2: Smaller models and faster training. In International conference on machine learning, pages 10096–10106. PMLR, 2021. 2

    2021

  24. [32]

    Linformer: Self-attention with linear complexity

    Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768, 2020. 3

    2006 arXiv

  25. [33]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference on computer vision , p...

    2021

  26. [34]

    Cbam: Convolutional block attention module

    Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018. 3

    2018

  27. [35]

    Rethinking and improving relative posi- tion encoding for vision transformer

    Kan Wu, Houwen Peng, Minghao Chen, Jianlong Fu, and Hongyang Chao. Rethinking and improving relative posi- tion encoding for vision transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10033–10041, 2021. 4

    2021

  28. [36]

    Aggregated residual transformations for deep neural networks

    Saining Xie, Ross Girshick, Piotr Doll ´ar, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492–1500,

  29. [37]

    Focal modulation networks

    Jianwei Yang, Chunyuan Li, Xiyang Dai, and Jianfeng Gao. Focal modulation networks. Advances in Neural Information Processing Systems, 35:4203–4217, 2022. 1

    2022

  30. [38]

    Metaformer is actually what you need for vision

    Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, and Shuicheng Yan. Metaformer is actually what you need for vision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10819–10829, 2022. 2, 6, 7, 4, 5

    2022

  31. [39]

    Differ- entiable learning-to-group channels via groupable convolu- tional neural networks

    Zhaoyang Zhang, Jingyu Li, Wenqi Shao, Zhanglin Peng, Ruimao Zhang, Xiaogang Wang, and Ping Luo. Differ- entiable learning-to-group channels via groupable convolu- tional neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3542– 3...

    2019

  32. [40]

    • Firstly, we provide detailed explanations of our experi- mental setup, the specifics of the three PATConv blocks, and the different PartialNet variants

    Overview In this supplementary material, we present more explana- tions and experimental results. • Firstly, we provide detailed explanations of our experi- mental setup, the specifics of the three PATConv blocks, and the different PartialNet variants. • Secondly, we present a...

  33. [41]

    Clarifications on Experimental Setting Firstly, we provide the ImageNet-1k training and evalua- tion settings in Tab. 8. These settings can be used to repro- duce our main results in Figure 1 of the main paper. Differ- ent PartialNet variants vary in the magnitude of regulariz...

  34. [42]

    For the full comparison of the classification task on the ImageNet-1k Benchmark, please refer to Tab

    Full Comparison on the ImageNet-1k Benchmark and COCO Benchmark. For the full comparison of the classification task on the ImageNet-1k Benchmark, please refer to Tab. 10, which complements the results provided in Table 1 of the main pa- per. For the full Comparison of the obje...

  35. [43]

    The Complements of Ablation Studies Partial Attention vs. Classic Visual Attention:To further prove the superiority of our proposed PATConv, we present experiment results for the combination of our partial atten- tion and classic visual attention networks, and the results are ...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.