Pith. sign in

REVIEW 3 major objections 6 minor 32 references

Cross-Layer Cache Aggregation for Token Reduction in Ultra-Fine-Grained Image Recognition

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that caching pre-reduction features and feeding intermediate-layer CLS tokens to the classifier lets token-reduced vision transformers keep competitive ultra-fine-grained accuracy at keep rates as low as 10%.

desk verdict Solid, well-scoped efficiency paper for UFGIR; the CLCA combo clearly helps, but the paper never isolates the cache, so the headline 'cache recovers lost information' is under-supported. read the letter →

arxiv 2501.00243 v1 pith:ILKFTKGR submitted 2024-12-31 cs.CV

Edwin Arkel Rios , Jansen Christopher Yuanda , Vincent Leon Ghanz , Cheng-Wei Yu , Bo-Cheng Lai , Min-Chun Hu This is my paper · ORCID
classification cs.CV
keywords ultra-fine-grainedimagerecognitiontokenreductionvisiontransformercross-layercachefeatureaggregationefficientinferencefine-grainedvisualcategorizationregistertokens
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ultra-fine-grained image recognition needs high-resolution inputs and attention to tiny visual differences, which makes the quadratic cost of vision transformer attention painful. Token reduction cuts that cost by dropping or merging image-patch tokens, but it also throws away exactly the small details these tasks depend on. This paper tries to recover that lost information with two plug-and-play modules: a Cross-Layer Cache that stores pooled local features and a learned register token before reduction and re-inserts them afterwards, and a Cross-Layer Aggregation classification head that routes intermediate CLS features straight into the classifier. Across more than 2000 runs, the combined method, CLCA, consistently improves accuracy over the token-reduced baseline and lets the keep rate fall to 10% while staying competitive with state-of-the-art ultra-fine-grained models. If this works, it provides a cheap way to make efficient vision transformers accurate on detail-critical tasks.

What carries the argument

The load-bearing mechanism is the pair of cross-layer connections. The Cross-Layer Cache (CLC) stores, after each transformer encoder block, the global average pooling (GAP) of local patch tokens and the output of a learnable Cross-Layer Register (CLR) token appended at the end of the sequence; after a reduction point these stored tokens are appended to the reduced sequence, then the cache is emptied to avoid repetition. The Cross-Layer Aggregation (CLA) Head takes the CLS tokens from the end of each encoder group, applies batch normalization, a depth-wise convolution that mixes channels across layers, a nonlinearity, and a point-wise convolution that outputs class predictions. Together they reintroduce information that token reduction discarded and expose intermediate features to the classifier with negligible extra computation.

What would settle it

Run EViT at 10% keep rate on SoyGene with full CLCA, with the cache disabled, and with the cache populated by post-reduction features instead of pre-reduction features; if accuracies are nearly equal across all three, the claim that the cache recovers lost information is refuted.

Watch

Extended reading notes

Core claim

The central claim is that the information lost when a vision transformer drops tokens can be recovered by preserving compact summaries of pre-reduction features and by letting the classifier see intermediate layers directly. Concretely, after every encoder block the model caches a global-average-pooled vector of local patch tokens plus a learnable Cross-Layer Register token appended to the sequence; after token reduction at layers 4, 7, and 10, and before the final layer, these cached tokens are appended back into the stream. In parallel, the Cross-Layer Aggregation head collects the CLS token from each encoder group, normalizes them, mixes them channel-wise with a depth-wise convolution, and projects the result to class scores. The authors argue these connections act as skip connections that smooth optimization and give the classifier access to low-, middle-, and high-level features, which is exactly what fine-grained discrimination needs. The experimental claim is that CLCA raises the accuracy floor of token-reduced models across five datasets, nine backbones, and seven token-reduction schemes, making 10% keep rates viable.

Load-bearing premise

The load-bearing premise is that the pre-reduction GAP-pooled local features and the learned register token still contain the fine details the reduced sequence has lost, and that appending them after reduction neither misaligns the sequence nor injects noise.

Editorial extensions

If this is right

  • Attaching CLCA to a token-reduced ViT consistently improves accuracy across seven token-reduction schemes, so the recovery mechanism does not depend on how tokens are selected for removal.
  • With CLCA, the token keep rate can be dropped to 10%, cutting FLOPs sharply while keeping accuracy competitive with much costlier state-of-the-art methods (for example, CLCA at 10% keep rate uses 25.2 GFLOPs on Cotton versus 447.9 GFLOPs for TransFG).
  • The gains hold across nine pretrained backbones, including self-supervised and weakly supervised ones, indicating the modules are a general fix rather than a re-tuning of one architecture.
  • Because the added modules cost little extra compute, CLCA shifts the accuracy-versus-cost frontier of ultra-fine-grained recognition toward much cheaper high-resolution inference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would test the same cache-and-reinject recipe on other detail-critical tasks, such as medical image classification or document understanding, where aggressive token pruning also risks discarding the decisive pixels.
  • The design leaves open which cache entry matters more: a run that keeps only the GAP features, or only the CLR token, would show whether the learned register is essential or whether pooled local statistics alone can carry the lost information.
  • If 10% keep rates become reliable, the practical cost bottleneck for ultra-fine-grained recognition may move from transformer FLOPs to data-loading and memory bandwidth, so system-level co-design becomes the next lever.
  • The gradient-instability discussion suggests CLCA may also stabilize training of token-reduced transformers from scratch, which could be tested by comparing convergence curves with and without the modules under identical random seeds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Cross-Layer Cache Aggregation (CLCA), two add-on modules for Vision Transformers under token reduction for ultra-fine-grained image recognition: a Cross-Layer Aggregation (CLA) head that fuses CLS-token outputs from encoder groups via batch norm, depth-wise convolution, and point-wise convolution, and a Cross-Layer Cache (CLC) that stores global-average-pooled local features and a learnable Cross-Layer Register token after each encoder block and re-injects them after token reduction. The authors evaluate CLCA across 5 UFGIR leaf datasets, 9 pretrained backbones, 7 token-reduction methods, 5 keep rates, and 2 resolutions, reporting that it consistently improves accuracy and allows a 10% token keep rate with competitive accuracy at much lower FLOPs.

Significance. If the claims hold, CLCA is a practically useful, plug-and-play recipe for making aggressive token reduction viable in fine-grained recognition, with unusually broad empirical coverage (2000+ runs, multiple TR paradigms, code released). The main strengths are the breadth of settings and the reproducible code link. However, the paper does not isolate the contribution of the Cross-Layer Cache from the Cross-Layer Aggregation head, and the main state-of-the-art table lacks error bars and contains mixed results, so the central mechanism-specific claim is not yet established.

major comments (3)
  1. [Section V-B, Table I] Table I reports CLCA(10%) on Cotton as 55.6, below CSDNet's 57.9, and CLCA(10%) on SoyGlobal as 61.1, below SIM-Tr's 70.7; because the table has no error bars or standard deviations despite Section IV stating that results are averaged over three seeds, the reader cannot judge whether these gaps are significant or whether the 'competitive accuracy' claim holds at 10% KR across datasets. Please report per-seed results or standard deviations and state precisely on which datasets the 10% accuracy is competitive.
  2. [Section III-C, Fig. 4] No component-level ablation is reported: every CLCA configuration combines the CLA head and the CLC, so the accuracy recovery at low keep rates cannot be attributed to the cache. Fig. 4 is a gradient-magnitude trace, not an ablation, and it does not test the CLC mechanism. Please add experiments with (i) base TR model, (ii) base + CLA head only, (iii) base + CLC only, and (iv) base + both, at least for EViT on SoyLocal/SoyGene at KR=10%, 25%, and 50%; this is load-bearing because Section III-C and the abstract credit the CLC specifically with counteracting token-reduction information loss.
  3. [Section III-C] The explanation of why CLCA works is qualitative: the claim that gradients are 'significantly unstable' before adding the modules is not quantified, and the analogy to skip connections and deep supervision does not distinguish the CLC from the CLA head. Please either support the gradient claim with a quantitative comparison (e.g., gradient norm statistics with and without each module) or restrict the discussion to what the experiments actually show.
minor comments (6)
  1. [Section I] The phrase 'We refer to our full method method' contains a duplicated word; please correct it.
  2. [Throughout] The word 'specially' is used where 'especially' is intended in several places; please correct them.
  3. [Fig. 4 caption] The caption calls the gradient trace an 'Ablation'; rename it to 'Gradient analysis' to avoid confusion, since it does not ablate any component.
  4. [Section III-C] The sentence 'inspired by previous which suggests' is missing the noun 'work'; please revise it.
  5. [Section III-C] The CLC description does not state whether cached tokens receive positional embeddings or how their order is fixed when appended to the reduced sequence; please clarify this implementation detail.
  6. [Section IV] The paper claims 'more than 2000 runs' but does not give an exact count or a per-condition breakdown; please add a run count/breakdown or remove the number.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CLCA is an empirical architecture study whose gains are measured on external UFGIR benchmarks; no fitted quantity is relabeled as a prediction and no load-bearing result reduces to its own inputs.

full rationale

The paper's claims are empirical: the CLCA modules (CLA head and CLC) are described algorithmically, and their effectiveness is reported as measured top-1 accuracy on externally defined datasets (Yu et al.'s Cotton, SoyAgeing, SoyGlobal, SoyLocal, SoyGene) against baseline ViTs and SotA methods. There is no fitted law or calibrated parameter that is later presented as a prediction. The CLA head concatenates existing intermediate CLS features and applies convolutions; the CLC stores GAP-pooled local features and a learnable CLR token before reduction and re-appends them afterward. These are architectural definitions, not equations whose output is equivalent to an input statistic. The paper does not invoke a uniqueness theorem, does not rely on self-citations as load-bearing evidence, and does not smuggle in an ansatz via prior work: its citations to FFVT, registers, and skip-connection literature are external context, not the derivation itself. The reviewer-identified concern that the CLC is never ablated separately (Section V reports only the combined CLCA or a gradient trace in Fig. 4) is a real limitation of attribution, because the individual contribution of the cache versus the CLA head is not isolated. However, an omitted ablation is an experimental-support gap, not circularity: it does not make the reported result equivalent to the method's assumptions by construction. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The paper introduces two plug-in modules with learnable parameters and a learned register token. The only explicitly hand-set architectural constant reported is DWG=2 in the depth-wise convolution. No constants are fitted to derive a law; the central claim is empirical.

free parameters (2)
  • DWG (depth-wise convolution groups) = 2
    Hand-set constant in the CLA head, controls the channel-wise aggregation output size (Eq. 1).
  • Encoder group count g = 4 (layers 1-4, 5-7, 8-10, 11-12)
    Division of the 12-layer ViT into groups based on token reduction at layers 4, 7, 10 and before the last layer; chosen to match EViT's reduction schedule.
assumptions (4)
  • standard math Self-attention in Vision Transformers has O(N^2) complexity with respect to sequence length.
    Used in the introduction to motivate token reduction.
  • domain assumption Ultra-fine-grained image recognition benefits from higher-resolution images.
    Motivates the efficiency problem; cited from FGIR literature but not re-derived.
  • ad hoc to paper Token reduction discards information that later layers need; re-injecting cached features restores it without harming the model.
    This is the core premise of the CLC module; the paper provides empirical evidence but no formal guarantee.
  • domain assumption Skip connections smooth the optimization landscape and provide deep supervision.
    Used in Section V-C to explain why CLCA works; based on prior empirical observations.
invented entities (1)
  • Cross-Layer Register (CLR) token
    purpose: A learnable token appended to the sequence and cached after each encoder block; it is recovered after token reduction to supply cross-layer information to the classifier.
    New learnable artifact introduced by the paper; no external test discriminates it from a generic learned token, so its contribution can only be measured inside the proposed architecture.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cross-Layer Cache Aggregation for Token Reduction in Ultra-Fine-Grained Image Recognition." pith.science (2026). https://pith.science/paper/ILKFTKGR

@misc{pith2026250100243,
  author       = {Pith},
  title        = {Pith review of: Cross-Layer Cache Aggregation for Token Reduction in Ultra-Fine-Grained Image Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ILKFTKGR}},
  note         = {Machine review of arXiv:2501.00243}
}
read the original abstract

Ultra-fine-grained image recognition (UFGIR) is a challenging task that involves classifying images within a macro-category. While traditional FGIR deals with classifying different species, UFGIR goes beyond by classifying sub-categories within a species such as cultivars of a plant. In recent times the usage of Vision Transformer-based backbones has allowed methods to obtain outstanding recognition performances in this task but this comes at a significant cost in terms of computation specially since this task significantly benefits from incorporating higher resolution images. Therefore, techniques such as token reduction have emerged to reduce the computational cost. However, dropping tokens leads to loss of essential information for fine-grained categories, specially as the token keep rate is reduced. Therefore, to counteract the loss of information brought by the usage of token reduction we propose a novel Cross-Layer Aggregation Classification Head and a Cross-Layer Cache mechanism to recover and access information from previous layers in later locations. Extensive experiments covering more than 2000 runs across diverse settings including 5 datasets, 9 backbones, 7 token reduction methods, 5 keep rates, and 2 image sizes demonstrate the effectiveness of the proposed plug-and-play modules and allow us to push the boundaries of accuracy vs cost for UFGIR by reducing the kept tokens to extremely low ratios of up to 10\% while maintaining a competitive accuracy to state-of-the-art models. Code is available at: \url{https://github.com/arkel23/CLCA}

Figures

Figures reproduced from arXiv: 2501.00243 by the authors.

Figure 1
Figure 1. Accuracy vs FLOPs for DeiT3 [17] with EViT [15] token reduction [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. From top to bottom, on the left is the overview for a ViT with the proposed Cross-Layer Cache Aggregation (CLCA). A learnable CLS and Cross [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Ablation showing max magnitude of gradient across all layers of [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Distribution of accuracies for multiple keep rates (25, 50, and 70%) [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 25 canonical work pages

  1. [1]

    Fine-Grained Image Analysis with Deep Learning: A Survey,

    X.-S. Wei, Y .-Z. Song, O. Mac Aodha, J. Wu, Y . Peng, J. Tang, J. Yang, and S. Belongie, “Fine-Grained Image Analysis with Deep Learning: A Survey,” IEEE Transactions on Pattern Analysis and Machine Intelli- gence, pp. 1–1, 2021, conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence

    work page 2021

  2. [2]

    Benchmark Platform for Ultra-Fine-Grained Visual Categorization Beyond Human Perfor- mance,

    X. Yu, Y . Zhao, Y . Gao, X. Yuan, and S. Xiong, “Benchmark Platform for Ultra-Fine-Grained Visual Categorization Beyond Human Perfor- mance,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2021, pp. 10 265–10 275, iSSN: 2380-7504

    work page 2021

  3. [3]

    The Image Data and Backbone in Weakly Supervised Fine-Grained Visual Categoriza- tion: A Revisit and Further Thinking,

    S. Ye, Y . Wang, Q. Peng, X. You, and C. L. P. Chen, “The Image Data and Backbone in Weakly Supervised Fine-Grained Visual Categoriza- tion: A Revisit and Further Thinking,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 34, no. 1, pp. 2–16, Jan. 2024, conference Name: IEEE Transactions on Circuits and Systems for Video Technology

    work page 2024

  4. [4]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” arXiv:2010.11929 [cs] , Oct. 2020, arXiv: 2010.11929

    arXiv 2010

  5. [5]

    TransFG: A Transformer Architecture for Fine-Grained Recognition,

    J. He, J.-N. Chen, S. Liu, A. Kortylewski, C. Yang, Y . Bai, and C. Wang, “TransFG: A Transformer Architecture for Fine-Grained Recognition,” in Proceedings of the First MiniCon Conference , Feb. 2022

    work page 2022

  6. [6]

    Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

    J. Wang, X. Yu, and Y . Gao, “Feature Fusion Vision Transformer for Fine-Grained Visual Categorization,” in British Machine Vision Conference (BMVC), Jul. 2021, arXiv: 2107.02341

    work page Pith review arXiv 2021

  7. [7]

    Quantifying Attention Flow in Transform- ers,

    S. Abnar and W. Zuidema, “Quantifying Attention Flow in Transform- ers,” May 2020, arXiv:2005.00928 [cs]

    arXiv 2020

  8. [8]

    Efficient Fine- Grained Object Recognition in High-Resolution Remote Sensing Images From Knowledge Distillation to Filter Grafting,

    L. Wang, J. Zhang, J. Tian, J. Li, L. Zhuo, and Q. Tian, “Efficient Fine- Grained Object Recognition in High-Resolution Remote Sensing Images From Knowledge Distillation to Filter Grafting,” IEEE Transactions on Geoscience and Remote Sensing , vol. 61, pp. 1–16, 2023, conference Name: IEEE Transactions on Geoscience and Remote Sensing

    work page 2023

Show all 32 references
  1. [9]

    Fine- Grained Ship Detection in High-Resolution Satellite Images With Shape- Aware Feature Learning,

    B. Guo, R. Zhang, H. Guo, W. Yang, H. Yu, P. Zhang, and T. Zou, “Fine- Grained Ship Detection in High-Resolution Satellite Images With Shape- Aware Feature Learning,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 16, pp. 1914–1926, 202...

    1914

  2. [10]

    DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification,

    Y . Rao, W. Zhao, B. Liu, J. Lu, J. Zhou, and C.-J. Hsieh, “DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification,” Oct. 2021

    2021

  3. [11]

    Adaptive Token Sampling For Efficient Vision Transformers,

    M. Fayyaz, S. A. Koohpayegani, F. R. Jafari, S. Sengupta, H. R. V . Joze, E. Sommerlade, H. Pirsiavash, and J. Gall, “Adaptive Token Sampling For Efficient Vision Transformers,” Jul. 2022, arXiv:2111.15667 [cs]

    2022 arXiv

  4. [12]

    Learning to Merge Tokens in Vision Transformers,

    C. Renggli, A. S. Pinto, N. Houlsby, B. Mustafa, J. Puigcerver, and C. Riquelme, “Learning to Merge Tokens in Vision Transformers,” Feb. 2022, arXiv:2202.12015 [cs]

    2022 arXiv

  5. [13]

    Token Merging: Your ViT But Faster,

    D. Bolya, C.-Y . Fu, X. Dai, P. Zhang, C. Feichtenhofer, and J. Hoffman, “Token Merging: Your ViT But Faster,” Mar. 2023, arXiv:2210.09461 [cs]

    2023 arXiv

  6. [14]

    Patch Slimming for Efficient Vision Transformers,

    Y . Tang, K. Han, Y . Wang, C. Xu, J. Guo, C. Xu, and D. Tao, “Patch Slimming for Efficient Vision Transformers,” Apr. 2022

    2022

  7. [15]

    EViT: Expediting Vision Transformers via Token Reorganizations,

    Y . Liang, C. Ge, Z. Tong, Y . Song, J. Wang, and P. Xie, “EViT: Expediting Vision Transformers via Token Reorganizations,” Oct. 2021

    2021

  8. [16]

    Self- slimmed Vision Transformer,

    Z. Zong, K. Li, G. Song, Y . Wang, Y . Qiao, B. Leng, and Y . Liu, “Self- slimmed Vision Transformer,” Sep. 2022

    2022

  9. [17]

    DeiT III: Revenge of the ViT,

    H. Touvron, M. Cord, and H. J ´egou, “DeiT III: Revenge of the ViT,” Apr. 2022, arXiv:2204.07118 [cs]

    2022 arXiv

  10. [18]

    Learning Contrastive Self- Distillation for Ultra-Fine-Grained Visual Categorization Targeting Lim- ited Samples,

    Z. Fang, X. Jiang, H. Tang, and Z. Li, “Learning Contrastive Self- Distillation for Ultra-Fine-Grained Visual Categorization Targeting Lim- ited Samples,” IEEE Transactions on Circuits and Systems for Video Technology, pp. 1–1, 2024, conference Name: IEEE Transactions on Circu...

    2024

  11. [19]

    Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer,

    W. Zeng, S. Jin, W. Liu, C. Qian, P. Luo, W. Ouyang, and X. Wang, “Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, USA: IEEE, Jun. 2022, p...

    2022

  12. [20]

    Vision Transformers Need Registers,

    T. Darcet, M. Oquab, J. Mairal, and P. Bojanowski, “Vision Transformers Need Registers,” Oct. 2023

    2023

  13. [21]

    SIM-Trans: Structure Information Model- ing Transformer for Fine-grained Visual Categorization,

    H. Sun, X. He, and Y . Peng, “SIM-Trans: Structure Information Model- ing Transformer for Fine-grained Visual Categorization,” in Proceedings of the 30th ACM International Conference on Multimedia , ser. MM ’22. New York, NY , USA: Association for Computing Machinery, Oct. 202...

    2022

  14. [22]

    Fair Compar- ison: Quantifying Variance in Results for Fine-Grained Visual Catego- rization,

    M. Gwilliam, A. Teuscher, C. Anderson, and R. Farrell, “Fair Compar- ison: Quantifying Variance in Results for Fine-Grained Visual Catego- rization,” 2021, pp. 3309–3318

    2021

  15. [23]

    Training data-efficient image transformers & distilla- tion through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J ´egou, “Training data-efficient image transformers & distilla- tion through attention,” arXiv:2012.12877 [cs] , Jan. 2021, arXiv: 2012.12877

    2012 arXiv

  16. [24]

    ImageNet-21K Pretraining for the Masses,

    T. Ridnik, E. Ben-Baruch, A. Noy, and L. Zelnik, “ImageNet-21K Pretraining for the Masses,” Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks , vol. 1, Dec. 2021

    2021

  17. [25]

    An Empirical Study of Training Self- Supervised Vision Transformers,

    X. Chen, S. Xie, and K. He, “An Empirical Study of Training Self- Supervised Vision Transformers,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Montreal, QC, Canada: IEEE, Oct. 2021, pp. 9620–9629

    2021

  18. [26]

    Emerging Properties in Self-Supervised Vision Transform- ers,

    M. Caron, H. Touvron, I. Misra, H. Jegou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging Properties in Self-Supervised Vision Transform- ers,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2021, pp. 9630–9640, iSSN: 2380-7504

    2021

  19. [27]

    Masked Autoencoders Are Scalable Vision Learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Dollar, and R. Girshick, “Masked Autoencoders Are Scalable Vision Learners,” in 2022 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR) . New Orleans, LA, USA: IEEE, Jun. 2022, pp. 15 979–15 988

    2022

  20. [28]

    LAION-5B: An open large-scale dataset for training next generation image-text models,

    C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wight- man, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, P. Schramowski, S. Kundurthy, K. Crowson, L. Schmidt, R. Kaczmar- czyk, and J. Jitsev, “LAION-5B: An open large-scale dataset for training next generation im...

    2022

  21. [29]

    Visualizing the Loss Landscape of Neural Nets,

    H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein, “Visualizing the Loss Landscape of Neural Nets,” in Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc., 2018

    2018

  22. [30]

    Towards Understanding the Importance of Shortcut Connections in Residual Networks,

    T. Liu, M. Chen, M. Zhou, S. S. Du, E. Zhou, and T. Zhao, “Towards Understanding the Importance of Shortcut Connections in Residual Networks,” in Advances in Neural Information Processing Systems , vol. 32. Curran Associates, Inc., 2019

    2019

  23. [31]

    Densely Connected Convolutional Networks,

    G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , Jul. 2017, pp. 2261– 2269, iSSN: 1063-6919

    2017

  24. [32]

    Residual networks behave like ensembles of relatively shallow networks,

    A. Veit, M. Wilber, and S. Belongie, “Residual networks behave like ensembles of relatively shallow networks,” in Proceedings of the 30th International Conference on Neural Information Processing Systems, ser. NIPS’16. Red Hook, NY , USA: Curran Associates Inc., Dec. 2016, pp. 550–558

    2016

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.