REVIEW 32 cited by
ImageNet-21K Pretraining for the Masses
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
ImageNet-1K serves as the primary dataset for pretraining deep learning models for computer vision tasks. ImageNet-21K dataset, which is bigger and more diverse, is used less frequently for pretraining, mainly due to its complexity, low accessibility, and underestimation of its added value. This paper aims to close this gap, and make high-quality efficient pretraining on ImageNet-21K available for everyone. Via a dedicated preprocessing stage, utilization of WordNet hierarchical structure, and a novel training scheme called semantic softmax, we show that various models significantly benefit from ImageNet-21K pretraining on numerous datasets and tasks, including small mobile-oriented models. We also show that we outperform previous ImageNet-21K pretraining schemes for prominent new models like ViT and Mixer. Our proposed pretraining pipeline is efficient, accessible, and leads to SoTA reproducible results, from a publicly available dataset. The training code and pretrained models are available at: https://github.com/Alibaba-MIIL/ImageNet21K
Forward citations
Cited by 32 Pith papers
-
CanViT: Toward Active-Vision Foundation Models
CanViT is the first task- and policy-agnostic AVFM pretrained via passive-to-active dense latent distillation on 13.2M scenes and 1B random glimpses, achieving 38.5% ADE20K mIoU in one glimpse and 84.5% ImageNet-1k to...
-
Condensing Large-Scale Datasets Directly with Minimal Information Loss
CIM directly aligns data distributions to condense large-scale datasets with minimal information loss, achieving new SOTA results on ImageNet-1K distillation at IPC=10.
-
Identifying Latent Concepts and Structures for Generalized Category Discovery
CPF-GCD enforces low-rank compositional structure on vision backbone features via spatial primitive fields so that novel categories emerge as new activation patterns over a shared vocabulary of reusable visual primitives.
-
GVCCS: A Dataset for Contrail Identification and Tracking on Visible Whole Sky Camera Sequences
GVCCS is the first open dataset of ground-based visible all-sky camera video with instance-level contrail masks, temporal tracking, and flight IDs, plus Mask2Former baselines.
-
The Edge-on Galaxies in the DESI survey (EGIDE): sample building and photometry
The EGIDE project releases a tenfold larger catalogue of edge-on galaxies with griz photometry, stellar masses, redshifts and star formation rates, finding that red-sequence galaxies are thicker than blue-cloud ones a...
-
The Edge-on Galaxies in the DESI survey (EGIDE): sample building and photometry
EGIDE provides 149,215 edge-on galaxy candidates from DESI DR10 with homogeneous photometry, masses, and redshifts, ten times larger than EGIPS.
-
ExDet: Open-Domain Open-Vocabulary Detection with Cross-modal Extrapolation and Rectification
ExDet proposes a lightweight framework using text-guided extrapolation, detector-compatible rectification, and recalibrated proposals to achieve state-of-the-art open-domain open-vocabulary detection on multiple benchmarks.
-
VISReg: Variance-Invariance-Sketching Regularization for JEPA training
VISReg replaces covariance in VICReg-style objectives with sliced-Wasserstein sketching for JEPA training, claiming better OOD performance and resilience to collapse.
-
LV-OSD: Language-Vision-Complementary Open-Set Object Detection
The authors define the LV-OSD problem and introduce the LVDor dual-branch framework with TPDW dynamic weighting and PRM masking to align multimodal prompts for open-set detection.
-
Energy-Structured Low-Rank Adaptation for Continual Learning
E²-LoRA structures low-rank adaptations by energy concentration and ordering with dynamic rank allocation to achieve state-of-the-art continual learning.
-
Weierstrass Positional Encoding for Vision Transformers
WePE encodes 2D patch positions in Vision Transformers via Weierstrass elliptic functions on the complex plane to exploit double periodicity and derive relative positions algebraically.
-
Birds of a Feather Flock Together: Background-Invariant Representations via Linear Structure in VLMs
Exploiting linear structure in VLM embeddings, a synthetic-data pre-training method yields background-invariant representations that exceed 90% worst-group accuracy on Waterbirds even under 100% spurious correlation w...
-
Parameter-Efficient Adaptation of Pre-Trained Vision Foundation Models for Active and Passive Seismic Data Denoising
Adapting vision foundation models with LoRA and kurtosis-guided unsupervised test-time adaptation matches or exceeds domain-specific models for seismic denoising across multiple sites and unseen data.
-
Contrastive Semantic Projection: Faithful Neuron Labeling with Contrastive Examples
Using contrastive examples with vision-language models and a new CLIP-based scoring method called CSP produces more faithful and granular neuron labels than prior activation-only approaches.
-
StableTTA: Improving Vision Model Performance by Training-free Test-Time Adaptation Methods
StableTTA improves ImageNet-1K accuracy across 71 vision models by stabilizing logit aggregation under coherent-batch inference and enabling efficient single-forward-pass adaptation.
-
MePo: Meta Post-Refinement for Rehearsal-Free General Continual Learning
MePo refines pretrained backbones via meta-learning on constructed pseudo tasks and initializes a meta covariance matrix to enable robust second-order alignment, yielding 12-15% gains on CIFAR-100, ImageNet-R and CUB-...
-
Page image classification for content-specific data processing
Fine-tuned image classifiers, led by RegNetY-16GF at about 99% top-1 accuracy, can sort historical archaeological page scans into 11 content categories, and the authors release the dataset, code, and model weights.
-
Towards Continuous Home Cage Monitoring: An Evaluation of Tracking and Identification Strategies for Laboratory Mice
A real-time mouse tracking and ear-tag identity pipeline reports 95.28% identification accuracy and fewer ID switches than SLEAP and DeepLabCut on a 100-minute home-cage dataset.
-
Opto-ViT: Architecting a Near-Sensor Region of Interest-Aware Vision Transformer Accelerator with Silicon Photonics
A near-sensor vision transformer accelerator combines VCSEL-microring photonic matrix multiplication with region-of-interest patch pruning, reporting 100.4 KFPS/W and up to 84% energy savings.
-
Revisiting Audio-Visual Segmentation with Vision-Centric Transformer
Vision-derived object queries with prototype prompting achieve new state-of-the-art results on AVSBench audio-visual segmentation.
-
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
LanguageBind aligns video, infrared, depth, and audio to a frozen language encoder via contrastive learning on the new VIDAL-10M dataset, extending video-language pretraining to N modalities.
-
JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search
JetViT uses post-training attention search to hybridize full-attention ViTs with linear and window attention blocks, achieving up to 1.79x throughput gains on high-res images while preserving accuracy on DINOv3 and De...
-
H3Former: Hypergraph-based Semantic-Aware Aggregation via Hyperbolic Hierarchical Contrastive Loss for Fine-Grained Visual Classification
A hypergraph-based token-to-region aggregation plus a hyperbolic hierarchical contrastive loss yields reported state-of-the-art fine-grained classification accuracy on four benchmarks.
-
Revisiting Deepfake Detection: Chronological Continual Learning and the Limits of Generalization
A chronological continual learning study finds deepfake detectors retain past knowledge but generalize to future generators at near-random AUC around 0.5.
-
Smelly, dense, and spreaded: The Object Detection for Olfactory References (ODOR) dataset
The ODOR dataset contributes 38,116 fine-grained object annotations over 4,712 artworks, benchmarked with five detector families, to stress-test object detection on dense, occluded, and off-centre objects in historica...
-
MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices
MobileVLM achieves on-par performance with much larger vision-language models on standard benchmarks while delivering state-of-the-art inference speeds of 21.5 tokens per second on Snapdragon 888 CPU and 65.3 on Jetso...
-
Page image classifier fine-tuned on century-spanning archives of scanned documents for further content-specific processing
Fine-tuned RegNetY-16GF reaches 99.16% accuracy classifying 48k century-old Czech archival pages into 11 visual content types, beating a 75% hand-crafted feature baseline, with models and data released publicly.
-
HandyLabel: Towards Post-Processing to Real-Time Annotation Using Skeleton Based Hand Gesture Recognition
HandyLabel enables real-time data annotation by mapping hand gestures to labels, with ResNet50 on skeleton-preprocessed HaGRID data reaching 0.923 F1-score and 88.9% of 46 study participants preferring it to tradition...
-
CascadeFormer: A Family of Two-stage Cascading Transformers for Skeleton-based Human Action Recognition
A masked-pretrained skeleton transformer with a second fine-tuning transformer and cross-attention fusion reaches 94.66% on Penn Action, 91.16% on N-UCLA, and 81.01%/88.17% on NTU RGB+D 60 cross-subject/cross-view.
-
Page image classification for content-specific data processing
Develops an image classification system for historical document pages to support content-specific analysis pipelines.
-
4th Workshop on Maritime Computer Vision (MaCVi): Challenge Overview
The report overviews five maritime computer vision benchmark challenges, their datasets, protocols, quantitative results, and top team approaches from the MaCVi 2026 workshop.
-
ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluation
ViT-ProtoNet, a Prototypical Network with a ViT-Small encoder, is reported to reach 95-97% 5-shot accuracy on three benchmarks and 81.88% on FC100, but the evaluation lacks critical baselines.
Discussion (0). Sign in to comment.