Pith. sign in

REVIEW 10 cited by

ImageNet-21K Pretraining for the Masses

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2104.10972 v4 pith:522CHFUV submitted 2021-04-22 cs.CV cs.LG

Tal Ridnik , Emanuel Ben-Baruch , Asaf Noy , Lihi Zelnik-Manor This is my paper
classification cs.CVcs.LG
keywords pretrainingimagenet-21kmodelsavailabledatasetefficienttaskstraining
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

ImageNet-1K serves as the primary dataset for pretraining deep learning models for computer vision tasks. ImageNet-21K dataset, which is bigger and more diverse, is used less frequently for pretraining, mainly due to its complexity, low accessibility, and underestimation of its added value. This paper aims to close this gap, and make high-quality efficient pretraining on ImageNet-21K available for everyone. Via a dedicated preprocessing stage, utilization of WordNet hierarchical structure, and a novel training scheme called semantic softmax, we show that various models significantly benefit from ImageNet-21K pretraining on numerous datasets and tasks, including small mobile-oriented models. We also show that we outperform previous ImageNet-21K pretraining schemes for prominent new models like ViT and Mixer. Our proposed pretraining pipeline is efficient, accessible, and leads to SoTA reproducible results, from a publicly available dataset. The training code and pretrained models are available at: https://github.com/Alibaba-MIIL/ImageNet21K

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GVCCS: A Dataset for Contrail Identification and Tracking on Visible Whole Sky Camera Sequences

    cs.CV 2025-07 conditional novelty 7.0 of 10

    GVCCS is the first open dataset of ground-based visible all-sky camera video with instance-level contrail masks, temporal tracking, and flight IDs, plus Mask2Former baselines.

  2. The Edge-on Galaxies in the DESI survey (EGIDE): sample building and photometry

    astro-ph.GA 2026-06 unverdicted novelty 6.0 of 10

    The EGIDE project releases a tenfold larger catalogue of edge-on galaxies with griz photometry, stellar masses, redshifts and star formation rates, finding that red-sequence galaxies are thicker than blue-cloud ones a...

  3. Towards Continuous Home Cage Monitoring: An Evaluation of Tracking and Identification Strategies for Laboratory Mice

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A real-time mouse tracking and ear-tag identity pipeline reports 95.28% identification accuracy and fewer ID switches than SLEAP and DeepLabCut on a 100-minute home-cage dataset.

  4. Opto-ViT: Architecting a Near-Sensor Region of Interest-Aware Vision Transformer Accelerator with Silicon Photonics

    cs.AR 2025-07 conditional novelty 6.0 of 10

    A near-sensor vision transformer accelerator combines VCSEL-microring photonic matrix multiplication with region-of-interest patch pruning, reporting 100.4 KFPS/W and up to 84% energy savings.

  5. Revisiting Audio-Visual Segmentation with Vision-Centric Transformer

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Vision-derived object queries with prototype prompting achieve new state-of-the-art results on AVSBench audio-visual segmentation.

  6. H3Former: Hypergraph-based Semantic-Aware Aggregation via Hyperbolic Hierarchical Contrastive Loss for Fine-Grained Visual Classification

    cs.CV 2025-11 conditional novelty 5.0 of 10

    A hypergraph-based token-to-region aggregation plus a hyperbolic hierarchical contrastive loss yields reported state-of-the-art fine-grained classification accuracy on four benchmarks.

  7. Revisiting Deepfake Detection: Chronological Continual Learning and the Limits of Generalization

    cs.LG 2025-08 conditional novelty 5.0 of 10

    A chronological continual learning study finds deepfake detectors retain past knowledge but generalize to future generators at near-random AUC around 0.5.

  8. Smelly, dense, and spreaded: The Object Detection for Olfactory References (ODOR) dataset

    cs.CV 2025-07 conditional novelty 5.0 of 10

    The ODOR dataset contributes 38,116 fine-grained object annotations over 4,712 artworks, benchmarked with five detector families, to stress-test object detection on dense, occluded, and off-centre objects in historica...

  9. CascadeFormer: A Family of Two-stage Cascading Transformers for Skeleton-based Human Action Recognition

    cs.CV 2025-08 conditional novelty 4.0 of 10

    A masked-pretrained skeleton transformer with a second fine-tuning transformer and cross-attention fusion reaches 94.66% on Penn Action, 91.16% on N-UCLA, and 81.01%/88.17% on NTU RGB+D 60 cross-subject/cross-view.

  10. ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluation

    cs.CV 2025-07 reject novelty 2.0 of 10

    ViT-ProtoNet, a Prototypical Network with a ViT-Small encoder, is reported to reach 95-97% 5-shot accuracy on three benchmarks and 81.88% on FC100, but the evaluation lacks critical baselines.

Pith tools